-- Logs begin at Mon 2022-09-26 17:00:18 UTC, end at Mon 2022-09-26 18:17:12 UTC. -- Sep 26 17:44:11.164841 dv-cinder-859318-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 26 17:44:11.172169 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 26 17:44:11.174166 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 26 17:44:11 2022] *** Sep 26 17:44:11.174166 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 26 17:44:11.174166 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 26 17:44:11.174166 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: nodename: dv-cinder-859318-1-iscsi Sep 26 17:44:11.174166 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: machine: x86_64 Sep 26 17:44:11.174166 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: clock source: unix Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: pcre jit disabled Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: detected number of CPU cores: 4 Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: current working directory: / Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: detected binary path: /usr/bin/uwsgi-core Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: your processes number limit is 47680 Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: your memory page size is 4096 bytes Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: detected max file descriptor number: 2048 Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: lock engine: pthread robust mutexes Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: thunder lock: enabled Sep 26 17:44:11.174554 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 26 17:44:11.175120 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 26 17:44:11.216464 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: Python main interpreter initialized at 0x55b38165c9c0 Sep 26 17:44:11.216464 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: python threads support enabled Sep 26 17:44:11.216464 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: your server socket listen backlog is limited to 100 connections Sep 26 17:44:11.216464 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: your mercy for graceful operations on workers is 90 seconds Sep 26 17:44:11.216859 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: mapped 403077 bytes (393 KB) for 2 cores Sep 26 17:44:11.216937 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: *** Operational MODE: preforking *** Sep 26 17:44:11.217206 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: *** uWSGI is running in multiple interpreter mode *** Sep 26 17:44:11.217206 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: spawned uWSGI master process (pid: 81395) Sep 26 17:44:11.217422 dv-cinder-859318-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 26 17:44:11.218998 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 1 (pid: 81409, cores: 1) Sep 26 17:44:11.218998 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 2 (pid: 81410, cores: 1) Sep 26 17:44:11.219590 dv-cinder-859318-1-iscsi devstack@keystone.service[81395]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 26 17:44:15.528388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:44:15.528388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: from cryptography.utils import int_from_bytes Sep 26 17:44:15.529169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:44:15.529169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: from cryptography.utils import int_from_bytes Sep 26 17:44:15.529605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:44:15.529605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: from cryptography.utils import int_from_bytes Sep 26 17:44:15.529757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:44:15.529757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: from cryptography.utils import int_from_bytes Sep 26 17:44:15.579601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 17:44:15.579790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 17:44:15.579964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] command line args: [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 17:44:15.580232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 17:44:15.580410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ================================================================================ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 17:44:15.580539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 17:44:15.580661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] admin_token = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.580745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 17:44:15.580912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] command line args: [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 17:44:15.580999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config_dir = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.581096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.581171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 17:44:15.581278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] config_source = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.581359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ================================================================================ {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 17:44:15.581460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] debug = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.581590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] admin_token = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.581666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.581892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] config_dir = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] config_source = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] debug = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.582776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_config_append = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_dir = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.583894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.584456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.584541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.584675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_config_append = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.584757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.584836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.584958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_dir = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_file = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.585941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.586934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] publish_errors = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.587955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] publish_errors = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.588842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] standard_threads = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_journal = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] standard_threads = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.589975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_json = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_stderr = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] use_syslog = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] use_journal = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.590789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.590887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] use_json = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.591033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] use_stderr = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.591110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.591210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] use_syslog = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.591306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.591442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:44:15.591548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.591701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.591784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.591946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.592949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.593996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.594980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.595998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.596921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.597885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.598808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.599001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.599078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.599233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.599317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.600981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.601964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.602922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.603873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.604962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.605974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.606948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.607100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.607177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.608988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.609956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.610935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.611884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.612850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.613960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.614998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.615255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.615876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.615968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.616976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.617920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.external = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.external = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.password = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.618987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] auth.token = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.password = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619695 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] auth.token = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.619928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.620870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.621984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.622984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.623966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.624988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.625934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.626931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.627966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.628920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.629862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.630785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.631349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.631492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.631703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.631959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.632208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.632480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.632680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.632759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.632950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.633999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.634955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.635904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.636856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.637976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.638977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.639945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.640788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.641995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642085 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.642989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.643966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.644936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.645962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.646121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.646227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.646452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.647865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.648987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.649993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.650997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.651957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.652998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.653967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.654139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.654389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.654469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.654622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.654906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.654991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.655135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.655358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.655438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.655619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.655850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.656049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.656290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.656376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.656544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.656758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.656836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] role.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.657956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] role.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.658927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.659906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.660136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.660251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.660369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.660606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.660691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.660850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.661892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.662660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.663984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.664085 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.664333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.664611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.664688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.664844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.664983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.665080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.665342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.665420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.665605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.665752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.665850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.666914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.667066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.667308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.667386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.667543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.667704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.667792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.668974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.669995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.670087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 17:44:15.670316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.670688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.671087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.671453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.671825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.672205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.672594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.672970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.673377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.673767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.674139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.674525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.674932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.675323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.675575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.675823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.676066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.676320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.676604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.676842 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.677091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.677362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.677725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.connection = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.678747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.678993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.679246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.679498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.679740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.679983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.680226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.680467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.680719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.680960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.681209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.681449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.681704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.681945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.682194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.682428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.682670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.682945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:44:15.683117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 17:44:15.700200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.700434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.700667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.708006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.708244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.708474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.709635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.717314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.738366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.738591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.743776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.744012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.792939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.793085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.842255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.842480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.842592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.842719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:15.999975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.000111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.000204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.000336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.000552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.024544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.024725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.024818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.024897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.024969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.025983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.036981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.037168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.049678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.049808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.049949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.050963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.051048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.051220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.051443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:44:16.068205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.068909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:44:16.068984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.069126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:44:16.069334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.069451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.069535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:44:16.069653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:44:16.069802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.069925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.070064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.070181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.070322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.070440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.070866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.070995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.080491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.080592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.080766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.080849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.081045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.081127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.081330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.081414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.088376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.088476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.091288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.091373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.091578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.091661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.091865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.091960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.092959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.093849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.094036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.094123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.117688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.123860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.123962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.123962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.124108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.124108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.124233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.124353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.124619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.125098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.125389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.125663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.131329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.131432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.132195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.132482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.137046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.137346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.137636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.137929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.138116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.138207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.138413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.138503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.138703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.138794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.139024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.139120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.139302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.139577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.139848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.140116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.154491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.154769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.155629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.155910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.158620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.159765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.161845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.162667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.163611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.163906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.166880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.168106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.169978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.171212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.173157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.173457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.173739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.174019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.174294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.174383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.174572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.174664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.174946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.175229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.175509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.175787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.190658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.190978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.191248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.191512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.191868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.192157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.192431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.192694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.200059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.200197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.201573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.201893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.203096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.203376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.203488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.203693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.206068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.206357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.206446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.206641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.206742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.206947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.207038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.207228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.207352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.207627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.218647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.218793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.218957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.219110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.219237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.219373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.219498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.219652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.219845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.220032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.220165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.220316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.220443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.220944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.236370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.236904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.239295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.239839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.242718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.243333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.244189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.244775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.245672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.245940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.246229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.246588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.246726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.246982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.252707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.253322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.254168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.254770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.257154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.257554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.257845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.257979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.258129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.258290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.258578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.258874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.265954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.266249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.266531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.266685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.266949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.267234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.273493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.273778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.274063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.274319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:44:16.274448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.274606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.274741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.274889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.275892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.276950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.277077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.277205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.277333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.277507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.277776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:44:16.312217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b38165c9c0 pid: 81410 (default app) Sep 26 17:44:16.312991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b38165c9c0 pid: 81409 (default app) Sep 26 17:44:16.315478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7529-8f34-4af3-914b-c306b4a48dff None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.315767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7529-8f34-4af3-914b-c306b4a48dff None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.316040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be7529-8f34-4af3-914b-c306b4a48dff None None] PATH_INFO: `/v3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.320371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1/1] 10.222.0.72 () {50 vars in 791 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:23.172555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.173091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.173091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.177401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1/2] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:23.182769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.183365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.183365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.254785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: INFO dbcounter [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] Registered counter for database keystone Sep 26 17:44:23.274464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG oslo_db.sqlalchemy.engines [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 17:44:23.280662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writer thread running {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 17:44:23.471827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG passlib.handlers.bcrypt [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81410) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 17:44:23.472219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG passlib.handlers.bcrypt [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81410) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 17:44:23.531036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:23.532588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:23.661020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 2/3] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 17:44:24.125059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:24.194277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: INFO dbcounter [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] Registered counter for database keystone Sep 26 17:44:24.216245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG oslo_db.sqlalchemy.engines [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81409) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 17:44:24.222315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writer thread running {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 17:44:24.341294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf1189-8d44-4507-b722-81838e1f18aa None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:24.345547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:24.346175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.346453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.346732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.355758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:24.406950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 17:44:24.407289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95588f-a8fc-4180-a046-5faf78bf7662 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:24.417234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 2/4] 10.222.0.72 () {58 vars in 1324 bytes} [Mon Sep 26 17:44:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 17:44:24.423175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:24.440918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47513e-12e0-41f1-ad1c-e537e4822d65 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:24.446315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:24.447453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.447739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.448013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] PATH_INFO: `/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.449721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] RBAC: Authorizing `identity:get_project(project_id=bbe873ec2e4b4dc7b12dd87b247ac3dc)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:24.502160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 17:44:24.502506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b1518-b9c6-4dc1-912e-2348b0462f98 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:24.507706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 3/5] 10.222.0.72 () {56 vars in 1207 bytes} [Mon Sep 26 17:44:24 2022] GET /identity/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.639009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b12b-f69e-49c1-b0e6-e3784a4cba44 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.639555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b12b-f69e-49c1-b0e6-e3784a4cba44 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.639555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b12b-f69e-49c1-b0e6-e3784a4cba44 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.643726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 3/6] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:26.650066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.650474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.650617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.671281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:26.672779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:26.680631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 4/7] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 17:44:26.958062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b12b-f69e-49c1-b0e6-e3784a4cba44 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.970323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b12b-f69e-49c1-b0e6-e3784a4cba44 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.974425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24d02-67b8-44e9-9aab-081e96a9b361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.975054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24d02-67b8-44e9-9aab-081e96a9b361 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.975338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24d02-67b8-44e9-9aab-081e96a9b361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.976164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24d02-67b8-44e9-9aab-081e96a9b361 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.984268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24d02-67b8-44e9-9aab-081e96a9b361 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.986026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24d02-67b8-44e9-9aab-081e96a9b361 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.995384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 4/8] 10.222.0.72 () {58 vars in 1324 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 17:44:27.001202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.010461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636ed3d-b1b2-4854-8626-f41783cf021b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.014911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18b460-f8ea-4964-b5c7-9d023012998a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.015538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18b460-f8ea-4964-b5c7-9d023012998a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.015821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18b460-f8ea-4964-b5c7-9d023012998a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.016097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18b460-f8ea-4964-b5c7-9d023012998a admin admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.017395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18b460-f8ea-4964-b5c7-9d023012998a admin admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:27.019265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18b460-f8ea-4964-b5c7-9d023012998a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:27.020844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 5/9] 10.222.0.72 () {56 vars in 1198 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:29.289943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abca9be-10bf-4899-9a4e-e3d0e73e8710 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.290468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abca9be-10bf-4899-9a4e-e3d0e73e8710 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.290468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abca9be-10bf-4899-9a4e-e3d0e73e8710 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.291969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 5/10] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:29.298076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f1840-0e79-4c46-8eaf-38f5c79c8dbb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.298585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f1840-0e79-4c46-8eaf-38f5c79c8dbb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.298678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f1840-0e79-4c46-8eaf-38f5c79c8dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.317700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-785f1840-0e79-4c46-8eaf-38f5c79c8dbb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:29.318938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-785f1840-0e79-4c46-8eaf-38f5c79c8dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:29.403505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 6/11] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:29.405238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c72630-ebb6-4a64-b583-538b7064352a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.405533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c72630-ebb6-4a64-b583-538b7064352a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.405815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c72630-ebb6-4a64-b583-538b7064352a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.407181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 6/12] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:29.412314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.412605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.412906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.431740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:29.432944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:29.449873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:29.502322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 7/13] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:29.675188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c72630-ebb6-4a64-b583-538b7064352a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:29.770590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:29.817537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c72630-ebb6-4a64-b583-538b7064352a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:29.851195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:29.852045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.852326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.852606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.863431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:29.871993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296b3b-2be2-4a12-9739-b29baaaa562a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:29.873793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:29.874679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:29.875546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 7/14] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:29.880654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:29.907165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:29.907785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.908069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.908346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.915595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:29.931397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:29.982167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:29.983779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:29.986942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e3346-3e7f-4ad9-93a2-725c764bda5f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:30.021432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:30.022039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:30.022444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:30.022724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:30.023435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:30.023762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:30.025289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:30.037039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 8/15] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:30.043595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:30.083527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 8/16] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:44:30.089172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:30.136223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:30.141388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:30.143171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f642e6d-cf4f-4513-90d8-506e68f5c9d1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:30.176754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:30.177393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:30.177678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:30.177954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:30.180101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:30.181606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:30.183492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 9/17] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:30 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:30.188651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:30.188766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44842c91-f539-47fb-9fb3-ba4fbd2d8ffb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:30.221849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:30.222481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:30.222763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:30.223069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:30.233887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:30.235403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:30.254019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:30.254972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:30.255975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:30.255975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:30.255975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:30.255975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 9/18] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:30.261398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:30.291589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea73b4-2fd6-4f1a-b3fb-e4e734fb2541 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:30.327089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:30.327727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:30.328013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:30.328293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:30.329506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:30.330995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:30.332576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 10/19] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:30.338479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:30.365904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc4fe9-9488-4506-8d67-9ac52894c67a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:30.369089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:30.370425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:30.401699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:30.402300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:30.402576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:30.402866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:30.403497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:30.403797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:30.405185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f571f61-9030-48a5-baf4-c8e8ad17c499 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:30.421741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 10/20] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:30.442742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e9e93-9f07-4fee-8737-d1edf8c46db5 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:30.481677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:30.482312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:30.482592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:30.482890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:30.483638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.domain.id=default, user.id=91cdfaf440314ad2b1f94da968a2f0eb) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:30.483947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:30.485439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d8e4a-2c61-4816-8f19-f8d325a4d914 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:30.496339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 11/21] 10.222.0.72 () {56 vars in 1322 bytes} [Mon Sep 26 17:44:30 2022] GET /identity/v3/role_assignments?user.id=91cdfaf440314ad2b1f94da968a2f0eb&scope.domain.id=default&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:33.065179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2a183-d11e-4e73-be82-3bc36a2682e5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.065721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2a183-d11e-4e73-be82-3bc36a2682e5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.065721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2a183-d11e-4e73-be82-3bc36a2682e5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.067109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 11/22] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:33.074004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.074510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.074613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.099396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG passlib.handlers.bcrypt [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81409) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 17:44:33.099787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG passlib.handlers.bcrypt [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81409) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 17:44:33.163941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:33.165582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b3b5c18e-ae30-4a4f-ab81-50c3bb444b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:33.283514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 12/23] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:33.320811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c19fe-8906-417f-bb32-09b75e01002c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.321130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c19fe-8906-417f-bb32-09b75e01002c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.321358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c19fe-8906-417f-bb32-09b75e01002c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.322706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 12/24] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:33.328387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264328-4970-4208-ba47-07187c43f68a None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.328677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264328-4970-4208-ba47-07187c43f68a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.328951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264328-4970-4208-ba47-07187c43f68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.356751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3b264328-4970-4208-ba47-07187c43f68a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:33.357964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3b264328-4970-4208-ba47-07187c43f68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:33.388179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.396462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae17fba-8575-4623-9898-6ed736f71bce None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.396747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae17fba-8575-4623-9898-6ed736f71bce None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.397020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae17fba-8575-4623-9898-6ed736f71bce None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.399821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 13/25] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:33.405110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246e91-2ce6-42b3-bd82-e986d55da542 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.405405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246e91-2ce6-42b3-bd82-e986d55da542 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.405682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246e91-2ce6-42b3-bd82-e986d55da542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.447906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e9246e91-2ce6-42b3-bd82-e986d55da542 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:33.449205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e9246e91-2ce6-42b3-bd82-e986d55da542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:33.477269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 13/26] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:33.571009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 14/27] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:33.603674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa0273-2c94-47d3-b4da-f57f2d814b19 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.603986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa0273-2c94-47d3-b4da-f57f2d814b19 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.604240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa0273-2c94-47d3-b4da-f57f2d814b19 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.605614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 14/28] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:33.611154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.611443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.611721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.639011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:33.640184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:33.744154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 15/29] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:33.848243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa0273-2c94-47d3-b4da-f57f2d814b19 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.915177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.965743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.991782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa0273-2c94-47d3-b4da-f57f2d814b19 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.075526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.076138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.076423 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.076701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.077052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f0bc5-7fa6-4cf7-bf7d-6c01db6768fd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.077426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.078807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.118882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.119632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.119923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.120183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.132829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:34.140356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:34.141226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:34.141978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:34.142559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 16/30] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:34.143867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.237259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:34.246258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 15/31] 10.222.0.72 () {60 vars in 1131 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/roles => generated 219 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:34.247283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.254880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296de0f-304a-4a3e-97ea-a680f7a88573 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.316329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.317002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.317332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.317608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.322084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:34.334260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.336413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:34.354550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:34.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:34.356320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:34.356320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:34.356320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:34.356320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 17/32] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:34.357693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.507512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec85291-590d-4751-816b-d5fdd135e217 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.533095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2aff8-bded-43b2-958b-4facb3777a33 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.550709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.551821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.552108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.552387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.553275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:34.553649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.555817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.568388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 16/33] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:34.570664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.581601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.582230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.582583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.582880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.583571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.585660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.595146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:34.669330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:34.692936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f2c2a-ff5b-4e66-8d4e-b969f744f4df None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.718896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 18/34] 10.222.0.72 () {60 vars in 1131 bytes} [Mon Sep 26 17:44:34 2022] POST /identity/v3/roles => generated 225 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:34.718896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.738476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.739280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.739642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.739991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.741932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:34.741932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.744221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5dc61-34e5-4503-a9c1-995ef1503ea3 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.769697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 17/35] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:34.771323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efd1a7-3be3-4126-ab54-83c640a4f1f6 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.771668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efd1a7-3be3-4126-ab54-83c640a4f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.772016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efd1a7-3be3-4126-ab54-83c640a4f1f6 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.773617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 18/36] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:34.779215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efd1a7-3be3-4126-ab54-83c640a4f1f6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.855273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baa98c-4261-4ecc-b318-4bf3fd13a920 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.901361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6a620-7218-4475-9cac-c1dfc7b4b023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.902017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6a620-7218-4475-9cac-c1dfc7b4b023 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.902555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6a620-7218-4475-9cac-c1dfc7b4b023 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.902555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6a620-7218-4475-9cac-c1dfc7b4b023 None admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.903923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6a620-7218-4475-9cac-c1dfc7b4b023 None admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.905495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6a620-7218-4475-9cac-c1dfc7b4b023 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.907151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 19/37] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:34 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:34.908353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd5f2-04bc-4a16-9872-9ca79206c67d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.908678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd5f2-04bc-4a16-9872-9ca79206c67d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.908918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fd5f2-04bc-4a16-9872-9ca79206c67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.912493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efd1a7-3be3-4126-ab54-83c640a4f1f6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.930675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-548fd5f2-04bc-4a16-9872-9ca79206c67d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:34.932261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-548fd5f2-04bc-4a16-9872-9ca79206c67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:34.957632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6c3b-1adc-4158-8c45-6c232f376d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.958432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6c3b-1adc-4158-8c45-6c232f376d6e None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.959143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6c3b-1adc-4158-8c45-6c232f376d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.959433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6c3b-1adc-4158-8c45-6c232f376d6e None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.960812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab6c3b-1adc-4158-8c45-6c232f376d6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.962283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab6c3b-1adc-4158-8c45-6c232f376d6e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.979953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:35.002571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 19/38] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:44:34 2022] POST /identity/v3/projects => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:35.003853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7606-947b-4caa-bb50-80462caa882b None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.004140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7606-947b-4caa-bb50-80462caa882b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.004416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7606-947b-4caa-bb50-80462caa882b None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.004512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:35.005665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 20/39] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:35.006438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e7606-947b-4caa-bb50-80462caa882b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:35.023952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 20/40] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:35.025780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.026233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.026967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.061590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:35.063841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:35.142176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e7606-947b-4caa-bb50-80462caa882b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:35.192430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:35.193139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.193421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.193690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.196184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:35.197701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:35.200899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 21/41] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:35.206740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:35.285804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:35.285967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 21/42] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:35.324394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:35.338555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d679b-981b-4613-bab7-2c705a12083c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:35.375046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:35.375725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.376009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.376291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] PATH_INFO: `/v3/domains/default/users/91cdfaf440314ad2b1f94da968a2f0eb/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.378492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=91cdfaf440314ad2b1f94da968a2f0eb, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:35.379973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:35.396366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 22/43] 10.222.0.72 () {58 vars in 1361 bytes} [Mon Sep 26 17:44:35 2022] PUT /identity/v3/domains/default/users/91cdfaf440314ad2b1f94da968a2f0eb/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:35.440653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee00acd-06ba-480d-b132-528d1fe8fc12 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:35.470986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:35.485136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:35.485812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.486079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.486359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.487567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:35.489032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:35.490936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 22/44] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:35.497122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:35.566531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:35.609764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f6c95-a75e-4a7a-813f-da1b65c106c1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:35.629460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:35.651307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850913a3-1112-43d5-b0bd-eea7e5bd2226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:35.652278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850913a3-1112-43d5-b0bd-eea7e5bd2226 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.652791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850913a3-1112-43d5-b0bd-eea7e5bd2226 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.653140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850913a3-1112-43d5-b0bd-eea7e5bd2226 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.653926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850913a3-1112-43d5-b0bd-eea7e5bd2226 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:35.655477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850913a3-1112-43d5-b0bd-eea7e5bd2226 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:35.685063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 23/45] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:44:35 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:35.686390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb16e3-83a7-4026-82a2-3c98db60167c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:35.728864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ee9a-83d2-4f45-b6d6-a8a716e3fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:35.729507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ee9a-83d2-4f45-b6d6-a8a716e3fbe7 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.729796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ee9a-83d2-4f45-b6d6-a8a716e3fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.730073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ee9a-83d2-4f45-b6d6-a8a716e3fbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.730826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91ee9a-83d2-4f45-b6d6-a8a716e3fbe7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:35.732349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91ee9a-83d2-4f45-b6d6-a8a716e3fbe7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:35.749210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 23/46] 10.222.0.72 () {60 vars in 1131 bytes} [Mon Sep 26 17:44:35 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:44:37.594748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67004-7342-4e0d-8323-c0bb51f19482 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.595316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67004-7342-4e0d-8323-c0bb51f19482 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.595399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67004-7342-4e0d-8323-c0bb51f19482 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.596709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 24/47] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:37.602545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e52d1-f021-4f1b-94c0-3060f835251a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.602978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e52d1-f021-4f1b-94c0-3060f835251a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.603149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e52d1-f021-4f1b-94c0-3060f835251a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.622312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-066e52d1-f021-4f1b-94c0-3060f835251a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:37.623546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-066e52d1-f021-4f1b-94c0-3060f835251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:37.671937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.692591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 24/48] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:37.889498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e831f5a-cc2f-4c96-8247-b5808dd89cce None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.889777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e831f5a-cc2f-4c96-8247-b5808dd89cce None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.890068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e831f5a-cc2f-4c96-8247-b5808dd89cce None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.891493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 25/49] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:37.896702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.897580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.897862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.916308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:37.917521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:37.958528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e831f5a-cc2f-4c96-8247-b5808dd89cce None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.988113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 25/50] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:38.064272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e831f5a-cc2f-4c96-8247-b5808dd89cce None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.097421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.098053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.098458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.098736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.108644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:38.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:38.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:38.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 26/51] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:38.120938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.219297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9b23-4d6b-41d0-95d7-620d7bf3c28c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.232539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.253487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.254099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.254383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.254661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.255360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.255673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.257628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.259801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.269423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 26/52] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.275861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.291145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.362718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8df67-293c-46b3-8954-d118a7d4873b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.376072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61788d3-69d4-4730-80a5-fe9ed97905f9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.397180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.397845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.398132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.398415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.399559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.401014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.402609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 27/53] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:38.408140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.409418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.410060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.410346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.410628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.411804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.413783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.419670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 27/54] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:38.420598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.466228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.508145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.512859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34270-401c-40fd-95e9-4a228a27f098 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.523355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d922256-a5bd-4406-bdac-0bf4e1502133 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.547963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a125c0-1121-4def-849d-2f4aa86b528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.548821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a125c0-1121-4def-849d-2f4aa86b528d None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.549115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a125c0-1121-4def-849d-2f4aa86b528d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.549541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a125c0-1121-4def-849d-2f4aa86b528d None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.550241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a125c0-1121-4def-849d-2f4aa86b528d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.552233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a125c0-1121-4def-849d-2f4aa86b528d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.560348 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.560988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.561278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.561555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.562667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.564141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.565718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 28/55] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.572142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.579978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 28/56] 10.222.0.72 () {60 vars in 1140 bytes} [Mon Sep 26 17:44:38 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:44:38.681958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ae02-99fe-4285-93b6-bd8b493b8e4c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.693084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.719867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.720520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.720811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.721095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.721842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.domain.id=default, user.id=91cdfaf440314ad2b1f94da968a2f0eb) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.722160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.723705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f458d8-5491-4dad-b80d-ce86af5ad8a7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.732426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 29/57] 10.222.0.72 () {56 vars in 1322 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/role_assignments?user.id=91cdfaf440314ad2b1f94da968a2f0eb&scope.domain.id=default&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:40.801050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd114be-ccd7-4ac8-997f-7de822ff277a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.801565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd114be-ccd7-4ac8-997f-7de822ff277a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.801647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd114be-ccd7-4ac8-997f-7de822ff277a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.803014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 29/58] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:40.808932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.809324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.809496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.828511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:40.829698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:40.894533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 30/59] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:41.164654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd114be-ccd7-4ac8-997f-7de822ff277a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.369333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd114be-ccd7-4ac8-997f-7de822ff277a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.402482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a2b6f-62b9-4c5a-a501-6048840e61cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.403134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2b6f-62b9-4c5a-a501-6048840e61cb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.403416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2b6f-62b9-4c5a-a501-6048840e61cb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.403689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2b6f-62b9-4c5a-a501-6048840e61cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.404870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a2b6f-62b9-4c5a-a501-6048840e61cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.406294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a2b6f-62b9-4c5a-a501-6048840e61cb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.407967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 30/60] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:41.413723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.509845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436d643-60b3-473b-b454-0c99299a05a4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.512121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:41.543868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d047bf29-5ad3-4e22-9de2-aa85e4cebd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.544472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047bf29-5ad3-4e22-9de2-aa85e4cebd4d None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.544755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047bf29-5ad3-4e22-9de2-aa85e4cebd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.545030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047bf29-5ad3-4e22-9de2-aa85e4cebd4d None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.545674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047bf29-5ad3-4e22-9de2-aa85e4cebd4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.547229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047bf29-5ad3-4e22-9de2-aa85e4cebd4d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.574522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 31/61] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:41 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:44.946405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e5672-436f-4d69-b7a9-e8487a5eae70 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.946949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e5672-436f-4d69-b7a9-e8487a5eae70 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.947067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e5672-436f-4d69-b7a9-e8487a5eae70 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.948367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 31/62] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:44.954275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0ff82-c792-4eaf-ac96-aa54c7d7d6f1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.955169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0ff82-c792-4eaf-ac96-aa54c7d7d6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.955650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0ff82-c792-4eaf-ac96-aa54c7d7d6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.983559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0aa0ff82-c792-4eaf-ac96-aa54c7d7d6f1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:44.984782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0aa0ff82-c792-4eaf-ac96-aa54c7d7d6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:45.048408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde5227-777f-43e8-950d-8d8d972239ee None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.048703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde5227-777f-43e8-950d-8d8d972239ee None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.049459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde5227-777f-43e8-950d-8d8d972239ee None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.051886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 32/63] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:45.058463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322ab1-4538-474e-8c62-03246be7a368 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.058759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322ab1-4538-474e-8c62-03246be7a368 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.059079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322ab1-4538-474e-8c62-03246be7a368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.086951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 32/64] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:45.100571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-da322ab1-4538-474e-8c62-03246be7a368 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:45.102190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-da322ab1-4538-474e-8c62-03246be7a368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:45.123355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.200637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 33/65] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:45.233210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8c6a9-a539-436f-ab9d-6db38122a94d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.233507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8c6a9-a539-436f-ab9d-6db38122a94d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.233777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8c6a9-a539-436f-ab9d-6db38122a94d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.235925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 33/66] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:45.242295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009095b-c842-44dd-8fce-8827d72b0273 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.242903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009095b-c842-44dd-8fce-8827d72b0273 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.243383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009095b-c842-44dd-8fce-8827d72b0273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.273944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-5009095b-c842-44dd-8fce-8827d72b0273 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:45.275515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5009095b-c842-44dd-8fce-8827d72b0273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:45.375512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 34/67] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:45.414783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8c6a9-a539-436f-ab9d-6db38122a94d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.508462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dedd-3723-4bd6-8513-64f5f94e5d40 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.509119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dedd-3723-4bd6-8513-64f5f94e5d40 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.509616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8dedd-3723-4bd6-8513-64f5f94e5d40 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.511336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 35/68] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:45.518299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.518827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.519413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.555018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.559492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8c6a9-a539-436f-ab9d-6db38122a94d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.560039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:45.561435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:45.617601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.618248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.618531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.618809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.633988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:45.640420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.641393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 34/69] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.642411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.668054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 36/70] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:45.669497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.682986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.765400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdaea7-c860-457b-b2b7-9addedc72de4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.803966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.805038 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.805560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.806025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.811021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5e7e8-5d6f-4e6e-ae3d-8a93b673fd59 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.817719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:45.824408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.825290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:45.826035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 35/71] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.826648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.841323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.854140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.854764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.855099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.855361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.856030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.856340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.857729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2847-c6a7-4ab5-8817-afb0fd84213f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.866251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 37/72] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.872581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18db410-ecb3-4ee1-9a55-f9fd03c110c8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.873105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18db410-ecb3-4ee1-9a55-f9fd03c110c8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.873576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18db410-ecb3-4ee1-9a55-f9fd03c110c8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.875941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 38/73] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:45.877125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18db410-ecb3-4ee1-9a55-f9fd03c110c8 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.936706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c413a-38f6-490b-98fa-48e679e54f64 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.957200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.984319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.985139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.985605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.986053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.001870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:46.010090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18db410-ecb3-4ee1-9a55-f9fd03c110c8 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:46.010814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.011769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:46.012506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:46.013386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 36/74] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.014667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.049275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.049834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.050103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.050406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.051164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.051759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.053948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e086574-8bd8-47c5-b3e1-75abbf1a4758 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.062031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 39/75] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.063360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.063651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.063925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.081857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:46.083070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:46.084805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.115859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6145ce-bbf7-4658-b14c-145a91a39ee4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.149464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.150090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.150377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.150540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 40/76] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:46.150648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.151281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.166610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.168382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.170087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 37/77] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.178292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.226243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.268240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c5842-ab5a-438d-8b18-cf5fb02bd7ed None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.281407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406dda-603a-4fc2-8a44-9a70d5abf132 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.305283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.305852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.306133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.306419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.310573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.317303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:46.319712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.320572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.320866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.321147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.321815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.322123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:46.322642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.323491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:46.324226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:46.324793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.324857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 41/78] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.324857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.332266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 38/79] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.336213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.433318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58661d-7b1a-48b8-a254-94a43d48dfa1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.442967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0b40c-05df-4749-bd0b-c467ca7ae422 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.457184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.469900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.470527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.470808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.471114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.472233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.473665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.475793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 42/80] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.478031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.478763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.479025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.479304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.481010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.545928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.558042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.559542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.587591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d1270-a0e0-45e4-9ca0-5455ae5dc586 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.621572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.622178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.622462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.622737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.623430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.623735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.625105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.634432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 43/81] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.635269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.659134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 39/82] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:46.659878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.695181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.743440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f46ba-f043-441e-80e5-969aef9fb603 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.749118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.759651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9caa16-88ca-4646-8900-c4819a1a3f6b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.778336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.779005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.779296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.779575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.780724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.782255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.783806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 44/83] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.784582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.795073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.795710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.795992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.796267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.797368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.798889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.800473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 40/84] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.804409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.887439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79dd4-0427-43c4-84f7-1477b15bf16a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.903098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d019be-dc53-4607-8d0f-862218cbc5a8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.907040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.921400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.922023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.922309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.922590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.936944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.937577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.937864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.938142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.939303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.940763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.942357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 41/85] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.943147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.979921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.011317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.012886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.052769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd17cc-69c3-4ed2-be06-31ffaad7e997 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.088749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.089384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.089666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.089943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.091096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.092552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.094378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 42/86] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.094991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.122176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 45/87] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:47.123045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.143790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.187581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.195825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054e761-9b4c-4ffc-aa1b-d42d75d94d1f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.223849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88965d3-322f-4c4e-adb0-fcac5440b8c6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.230301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.230941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.231227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.231501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.255676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.256855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.257143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.257422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.258075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.259570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.287935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 46/88] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:47 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:47.298387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.325083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.326587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.395429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.425807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28d75c-bfb7-4bed-8e01-8c1a43efb5d2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.435962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.445243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 43/89] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:47.446110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.477094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.478005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.478263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.478601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.555994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055acadd-0303-4902-a70f-cf7b3645aa03 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.565414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.567061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.599134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.599846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.600808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.601183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.601903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=0b672c7685154ab1a019a0a63150dcf1) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:47.602265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.603764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.609805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 44/90] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/role_assignments?user.id=0b672c7685154ab1a019a0a63150dcf1&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.610707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.681062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.689469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.714732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 47/91] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:47.719988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.742380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab85e0e-c9aa-46dc-a201-86c56a9f7801 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.783044 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.783943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.784378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.784803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.786234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.788621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.791050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 45/92] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.792025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.860371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072735c-0414-41a9-b2d6-8bf2e9cf8588 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.911011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b691749-bac0-4193-bc08-5828b3f975f3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.916318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.916976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.917261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.917545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.923344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.924826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.926532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 48/93] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.927429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.963502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.981152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.981801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.982083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.982357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] PATH_INFO: `/v3/projects/2a052524645a43918a55bcd8e2546b6a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.983814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] RBAC: Authorizing `identity:get_project(project_id=2a052524645a43918a55bcd8e2546b6a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.985281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.988433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 46/94] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/projects/2a052524645a43918a55bcd8e2546b6a => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.989093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.030458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.136311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e9b3-e979-4031-a50a-75ffdb934f43 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.178696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.179353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.179701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.180031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.180771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=91cdfaf440314ad2b1f94da968a2f0eb) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.181085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.182525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.184845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5885a5c-40e1-42a8-bed3-f3f628bce143 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.188799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 49/95] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/role_assignments?user.id=91cdfaf440314ad2b1f94da968a2f0eb&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:48.189612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.285145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.285692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.285980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.286264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.287041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] RBAC: Adding query filter params (role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=0b672c7685154ab1a019a0a63150dcf1) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.287884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.289365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fd554-9dd8-43a3-843a-6eb587ae540d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.301288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 47/96] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/role_assignments?user.id=0b672c7685154ab1a019a0a63150dcf1&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=238ba40fcefe4923bb27057b1074fbef => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:48.357419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedba20-721c-400c-a6dd-eac929c2ecb6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.380613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.427736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14312286-a480-46a9-897a-91918afcc097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.429082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14312286-a480-46a9-897a-91918afcc097 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.429371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14312286-a480-46a9-897a-91918afcc097 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.429651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14312286-a480-46a9-897a-91918afcc097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.430418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14312286-a480-46a9-897a-91918afcc097 None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=2a052524645a43918a55bcd8e2546b6a, user.id=0b672c7685154ab1a019a0a63150dcf1) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.430737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14312286-a480-46a9-897a-91918afcc097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.432303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14312286-a480-46a9-897a-91918afcc097 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.438700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 50/97] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:48 2022] GET /identity/v3/role_assignments?user.id=0b672c7685154ab1a019a0a63150dcf1&scope.project.id=2a052524645a43918a55bcd8e2546b6a&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:49.961328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa9c67-cd89-4635-941b-4c72d4e2ef44 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.961915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa9c67-cd89-4635-941b-4c72d4e2ef44 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.961915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa9c67-cd89-4635-941b-4c72d4e2ef44 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.963430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 48/98] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:49.971890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.972331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.972510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.007830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:50.009049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:50.102135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 51/99] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:50.645175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991071-ad04-43c8-9ba8-259d93f1b663 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.645571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991071-ad04-43c8-9ba8-259d93f1b663 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.645753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991071-ad04-43c8-9ba8-259d93f1b663 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.648198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 49/100] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:50.656882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.663278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.663747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.664032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.695717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:50.697452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:50.803598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.833124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 50/101] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:50.839393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e90f-1f92-4b5b-b0a6-dff108b93b0e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.847545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.903029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.903615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.903899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.904179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.931803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:50.936781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:50.937614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:50.938367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 52/102] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:50.946445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.104765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9737ba8-8677-483b-83b7-7e040f1ed8ef None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.143595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.144590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.145064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.145528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.146497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:51.147695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.149293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea9591-0ddf-40c2-a322-b5a133f35a6a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.162351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 51/103] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:51.168313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.218530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa768a-2168-4c10-8ece-dda84ab80390 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.219662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa768a-2168-4c10-8ece-dda84ab80390 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.221609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa768a-2168-4c10-8ece-dda84ab80390 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.224292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 52/104] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:51.234969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5a753-acbf-4090-82b0-694ba8ed03ce None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.235576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5a753-acbf-4090-82b0-694ba8ed03ce None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.236061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5a753-acbf-4090-82b0-694ba8ed03ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.262692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2ba5a753-acbf-4090-82b0-694ba8ed03ce None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:51.264276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2ba5a753-acbf-4090-82b0-694ba8ed03ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:51.323217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.344537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 53/105] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 26 17:44:51.361077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955d88b-ac9f-4f3e-99a3-db33dca2cd23 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.361689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955d88b-ac9f-4f3e-99a3-db33dca2cd23 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.362207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955d88b-ac9f-4f3e-99a3-db33dca2cd23 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.364078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 54/106] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:51.371556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.372104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.372579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.373413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfcfbc-d144-4c56-a2ad-36b5c1ae7c40 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.403094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:51.406165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:51.451171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.452180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.453418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.453902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] PATH_INFO: `/v3/users/2586d38983cf442a957e3ca9f03a6c3b` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.474527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] RBAC: Authorizing `identity:get_user(user_id=2586d38983cf442a957e3ca9f03a6c3b)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.476869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.478792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 53/107] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/users/2586d38983cf442a957e3ca9f03a6c3b => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:51.480783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.527458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.552987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 55/108] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:51.555690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.650200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e277dc-d636-4b93-829d-60e4341a9006 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.706336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a67ca-7763-442e-bd77-370ee6c6aeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.706940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a67ca-7763-442e-bd77-370ee6c6aeba None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.707744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a67ca-7763-442e-bd77-370ee6c6aeba None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.708306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a67ca-7763-442e-bd77-370ee6c6aeba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.709683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a67ca-7763-442e-bd77-370ee6c6aeba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.711513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a67ca-7763-442e-bd77-370ee6c6aeba None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.717720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 54/109] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:51.719030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d57058-89c8-4684-bfa8-9fcb90ad942d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.719326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d57058-89c8-4684-bfa8-9fcb90ad942d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.719612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d57058-89c8-4684-bfa8-9fcb90ad942d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.720921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 55/110] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:51.722005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d57058-89c8-4684-bfa8-9fcb90ad942d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.758207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c114a-4c8d-4f28-8b48-0c5c4b36fa2a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.773361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.811186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.812214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.813180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.813705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.877267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.894691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d57058-89c8-4684-bfa8-9fcb90ad942d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.908049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.910586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.932310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91625b15-0129-478a-9b2c-a94e1a2436dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.932930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91625b15-0129-478a-9b2c-a94e1a2436dd None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.933216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91625b15-0129-478a-9b2c-a94e1a2436dd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.933495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91625b15-0129-478a-9b2c-a94e1a2436dd None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.934171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91625b15-0129-478a-9b2c-a94e1a2436dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.935636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91625b15-0129-478a-9b2c-a94e1a2436dd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.961481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 56/111] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:44:51 2022] POST /identity/v3/projects => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:44:51.962619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.962933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.963211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.991299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:51.992551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:52.037861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 56/112] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:52.039130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.054012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.076978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 57/113] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:52.077987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.164077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.176904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26406a0-af42-4e02-a10d-3359290bf83b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.208454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd88ed-4dde-4c10-9ba5-432eedbcd88f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.222035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dba298-5311-44d1-8f5a-f99d6604258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.223072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba298-5311-44d1-8f5a-f99d6604258a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.223538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba298-5311-44d1-8f5a-f99d6604258a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.223993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dba298-5311-44d1-8f5a-f99d6604258a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.242545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dba298-5311-44d1-8f5a-f99d6604258a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-07dba298-5311-44d1-8f5a-f99d6604258a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:52.247915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.248750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:52.249545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:52.250108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.250772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.251157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.251502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.253786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 57/114] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.255063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c602-f791-4596-9414-7e1413a8bd9f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.255350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c602-f791-4596-9414-7e1413a8bd9f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.255628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1c602-f791-4596-9414-7e1413a8bd9f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.258351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 58/115] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:52.259218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c602-f791-4596-9414-7e1413a8bd9f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.268297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:52.275405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.276282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:52.277033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:52.277608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 58/116] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.278652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.345063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.382283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1c602-f791-4596-9414-7e1413a8bd9f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.398375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65c591-18dc-4d26-86f9-b3e02a8de490 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.418792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619784f0-daed-4057-90b2-bab2595dc23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.419458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619784f0-daed-4057-90b2-bab2595dc23f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.419748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619784f0-daed-4057-90b2-bab2595dc23f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.420027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619784f0-daed-4057-90b2-bab2595dc23f None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.421150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619784f0-daed-4057-90b2-bab2595dc23f None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.422602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619784f0-daed-4057-90b2-bab2595dc23f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.424288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 59/117] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.425403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.425688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.425967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.434915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.441991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.442860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.443859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.444459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.444988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:52.446204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:52.446419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.447092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.448722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.457957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 59/118] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.458785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.531241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 60/119] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:52.531241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.589857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c166c-2777-46b5-a247-47e5592dac2f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.627421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.627644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1453bff-e045-4211-8712-913386037529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.628223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1453bff-e045-4211-8712-913386037529 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.628676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1453bff-e045-4211-8712-913386037529 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.628951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1453bff-e045-4211-8712-913386037529 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.629794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1453bff-e045-4211-8712-913386037529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.630273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1453bff-e045-4211-8712-913386037529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.633363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1453bff-e045-4211-8712-913386037529 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.644941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 60/120] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.650880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1453bff-e045-4211-8712-913386037529 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.673322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785ee9-3add-4535-959f-604c3635e127 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.715298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.727251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.728501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.728936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.730006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.744261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:52.751166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.752179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:52.753127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:52.755748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 61/121] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.757494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.768213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1453bff-e045-4211-8712-913386037529 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.811235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.811835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.812127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.812407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.813144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] RBAC: Adding query filter params (role.id=b7d97d6c79bf4e0c90bce6aec6777332, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=2586d38983cf442a957e3ca9f03a6c3b) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.813458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.814962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.821841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 61/122] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/role_assignments?user.id=2586d38983cf442a957e3ca9f03a6c3b&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=b7d97d6c79bf4e0c90bce6aec6777332 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.831936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.883695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724a41-8cea-49aa-9f40-01069b5f81b8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.915641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.925739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.926451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.926741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.927090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.929217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.930880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.933051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 62/123] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.933659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.983761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a9bfb-f3d2-4b4d-800f-1a804337c3ee None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.004950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.037834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.038486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.038773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.039134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.040291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.041851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.043490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 62/124] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.048661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.106501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414fdfd-285e-4097-bbd8-0871c2d9f45e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.157501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.158953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.159524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.159980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.160850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.161233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4d7dd-e445-4a0e-8182-72cb9195ddcd None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.161529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.164208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.175976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 63/125] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.177557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.200495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.201295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.201670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.202054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.214250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:53.219975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:53.220828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:53.221604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 63/126] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:53.222230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.256548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.290490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.306162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc5f39-2e01-4d12-9311-c47750a1c550 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.343297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.343991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.344284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.344565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.345702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.347203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.349170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 64/127] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.350218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.369640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a4af-107f-4ddc-b5a6-80ac975e87f9 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.412467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.413249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.413539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.414350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.415581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.417462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.419822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 64/128] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.420624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.492813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b78e6-5dd4-448f-ae65-cb6b4d0ae2a0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.522597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.535661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.535954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3ed1f-7b0b-4ffe-a183-63a93d26f110 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.536321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.536609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.536888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.539504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.541087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.547212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 65/129] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.547212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.559472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.588659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.589282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.589567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.589845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.590520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.590853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.592219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.609127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 65/130] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.609127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.704646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48537c90-8dbb-4a94-aa0a-5f519091b1fc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.728601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f14f8-5006-4b8d-b8fa-77fce9049800 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.754905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.756234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.757123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.757856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/0b672c7685154ab1a019a0a63150dcf1/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.760925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] RBAC: Authorizing `identity:create_grant(project_id=ee6e468e90c34c0680f4b1a15b0f5055, user_id=0b672c7685154ab1a019a0a63150dcf1, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.763094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.771204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.772762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.773237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.773669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/91cdfaf440314ad2b1f94da968a2f0eb/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.776243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] RBAC: Authorizing `identity:create_grant(project_id=ee6e468e90c34c0680f4b1a15b0f5055, user_id=91cdfaf440314ad2b1f94da968a2f0eb, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.778343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.783942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 66/131] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:53 2022] PUT /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/0b672c7685154ab1a019a0a63150dcf1/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:53.785134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.822060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 66/132] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:53 2022] PUT /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/91cdfaf440314ad2b1f94da968a2f0eb/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:53.823817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.888174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.906529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.006787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9097a2-9674-4522-9e79-e411b04a559f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.054043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1577dc3-f573-449f-92ca-b35ed5d4b69e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.085630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.087268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.087598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.087944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] PATH_INFO: `/v3/projects/2a052524645a43918a55bcd8e2546b6a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.089964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] RBAC: Authorizing `identity:get_project(project_id=2a052524645a43918a55bcd8e2546b6a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.091717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.093628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 67/133] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/projects/2a052524645a43918a55bcd8e2546b6a => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:54.102048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.122095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.122325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.122548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.122739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.124627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.126435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.128586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 67/134] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:54.134341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.229088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89ddf-f798-44fe-b2e3-a667798a1c3d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.255690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.263860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe403342-9628-4918-8143-89cf65af6b45 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.266455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.275166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.276190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.276695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.277221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] PATH_INFO: `/v3/projects/2a052524645a43918a55bcd8e2546b6a/users/0b672c7685154ab1a019a0a63150dcf1/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.279583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a052524645a43918a55bcd8e2546b6a, user_id=0b672c7685154ab1a019a0a63150dcf1, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.281377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.300403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 68/135] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:54 2022] PUT /identity/v3/projects/2a052524645a43918a55bcd8e2546b6a/users/0b672c7685154ab1a019a0a63150dcf1/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:54.303956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8e142-d2d4-4f1f-8b10-94a455cf69d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.304615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8e142-d2d4-4f1f-8b10-94a455cf69d2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.304900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8e142-d2d4-4f1f-8b10-94a455cf69d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.305177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8e142-d2d4-4f1f-8b10-94a455cf69d2 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.306365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8e142-d2d4-4f1f-8b10-94a455cf69d2 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.307961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8e142-d2d4-4f1f-8b10-94a455cf69d2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.309709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 68/136] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:54.315700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.442438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bfec4-db70-4669-a04e-9eeba155040a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.497438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56958bac-dc11-4693-97fc-349ac99d545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.498188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958bac-dc11-4693-97fc-349ac99d545e None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.498555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958bac-dc11-4693-97fc-349ac99d545e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.498860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958bac-dc11-4693-97fc-349ac99d545e None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/0b672c7685154ab1a019a0a63150dcf1/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.501279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56958bac-dc11-4693-97fc-349ac99d545e None admin] RBAC: Authorizing `identity:create_grant(project_id=ee6e468e90c34c0680f4b1a15b0f5055, user_id=0b672c7685154ab1a019a0a63150dcf1, role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.502961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56958bac-dc11-4693-97fc-349ac99d545e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.528957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 69/137] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:54 2022] PUT /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/0b672c7685154ab1a019a0a63150dcf1/roles/238ba40fcefe4923bb27057b1074fbef => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:54.665718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c477-f2ff-4cd6-934a-322de9548fad None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.666062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c477-f2ff-4cd6-934a-322de9548fad None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.666347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c477-f2ff-4cd6-934a-322de9548fad None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.667892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 69/138] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:54.674460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.674460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.674460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.701025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:54.702586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:54.760782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.807595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 70/139] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:55.098683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c477-f2ff-4cd6-934a-322de9548fad None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.228193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248c477-f2ff-4cd6-934a-322de9548fad None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.263221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc4c2-d16a-4dc8-af04-6544d9b36287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.263863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc4c2-d16a-4dc8-af04-6544d9b36287 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.264148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc4c2-d16a-4dc8-af04-6544d9b36287 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.264672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc4c2-d16a-4dc8-af04-6544d9b36287 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.265518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bc4c2-d16a-4dc8-af04-6544d9b36287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.267001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bc4c2-d16a-4dc8-af04-6544d9b36287 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.268806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 70/140] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:55.274748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.340118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5709-fb7a-41a5-8340-ba2dd49250db None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.340445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5709-fb7a-41a5-8340-ba2dd49250db None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.340680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5709-fb7a-41a5-8340-ba2dd49250db None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.342209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 71/141] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:55.347069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.347353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.347629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.373387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:55.374509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:55.415065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5af1c-65bc-494d-8584-ecedf7ff38dc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.440353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.464297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.466743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.468096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.468575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.469473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.471964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.508451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 72/142] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:55.517582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 71/143] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:55 2022] POST /identity/v3/users => generated 283 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:55.916368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.048550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb348a-8546-48f5-a5f3-234fbbb37324 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.088497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.089130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.089404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.089675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.109094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:56.113963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:56.114864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:56.115749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 73/144] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:56.123971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.173246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.267519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b8be-68a3-41c0-9bfa-e6d8a97b9959 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.303475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.304200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.304364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.304623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.305347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:56.305663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.307098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.315708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 72/145] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.322482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.376323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.470818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7b4c-a599-4356-99c2-305ce317683a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.511985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.512624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.512901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.513172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] PATH_INFO: `/v3/users/2586d38983cf442a957e3ca9f03a6c3b` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.514430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] RBAC: Authorizing `identity:get_user(user_id=2586d38983cf442a957e3ca9f03a6c3b)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.515903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.517790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 74/146] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/users/2586d38983cf442a957e3ca9f03a6c3b => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.524008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.630734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.645119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6da0bf-67a5-40e0-8d89-36cd709106c0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.683647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7224460-fa59-4329-a7fe-354b708f5d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.684296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224460-fa59-4329-a7fe-354b708f5d03 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.684581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224460-fa59-4329-a7fe-354b708f5d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.684860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224460-fa59-4329-a7fe-354b708f5d03 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.685996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7224460-fa59-4329-a7fe-354b708f5d03 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.687511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7224460-fa59-4329-a7fe-354b708f5d03 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.689345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 73/147] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.697142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.836113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.840462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8384a-e9b2-4c5f-8d57-dc3d378f2513 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.878188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a37c10-e02d-43ec-ba5b-64fbd36d3d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.878889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a37c10-e02d-43ec-ba5b-64fbd36d3d83 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.879181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a37c10-e02d-43ec-ba5b-64fbd36d3d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.879460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a37c10-e02d-43ec-ba5b-64fbd36d3d83 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/2586d38983cf442a957e3ca9f03a6c3b/roles/b7d97d6c79bf4e0c90bce6aec6777332` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.881548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a37c10-e02d-43ec-ba5b-64fbd36d3d83 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee6e468e90c34c0680f4b1a15b0f5055, user_id=2586d38983cf442a957e3ca9f03a6c3b, role_id=b7d97d6c79bf4e0c90bce6aec6777332)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.883024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a37c10-e02d-43ec-ba5b-64fbd36d3d83 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.916206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 75/148] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:56 2022] PUT /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/users/2586d38983cf442a957e3ca9f03a6c3b/roles/b7d97d6c79bf4e0c90bce6aec6777332 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:57.794354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190172e6-0a16-45dd-bc9f-f53b8e2976ee None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.794903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190172e6-0a16-45dd-bc9f-f53b8e2976ee None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.794903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190172e6-0a16-45dd-bc9f-f53b8e2976ee None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.796297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 74/149] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:57.813486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fbb06-775f-42f6-8704-4d915946f334 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.814064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fbb06-775f-42f6-8704-4d915946f334 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.814064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fbb06-775f-42f6-8704-4d915946f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.845063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a09fbb06-775f-42f6-8704-4d915946f334 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:57.846919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a09fbb06-775f-42f6-8704-4d915946f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:57.946694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 76/150] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:57.960986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f932055-31e2-4492-8466-6e28b4177776 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.961264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f932055-31e2-4492-8466-6e28b4177776 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.961547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f932055-31e2-4492-8466-6e28b4177776 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.962951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 75/151] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:57.970207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67d65b-e50f-4554-87da-8d6d148da4c1 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.970207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67d65b-e50f-4554-87da-8d6d148da4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.970207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67d65b-e50f-4554-87da-8d6d148da4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.992581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6d67d65b-e50f-4554-87da-8d6d148da4c1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:57.993792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6d67d65b-e50f-4554-87da-8d6d148da4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:58.019795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bad076-85c1-4361-842d-40bddd785fa8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.020119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bad076-85c1-4361-842d-40bddd785fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.020349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bad076-85c1-4361-842d-40bddd785fa8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.022557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 76/152] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:58.027385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.027669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.027946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.054992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:58.056994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:58.112001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 77/153] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:58.150141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e8c5-15ff-4b0d-b990-d175f947afda None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.150458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e8c5-15ff-4b0d-b990-d175f947afda None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.150696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e8c5-15ff-4b0d-b990-d175f947afda None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.152378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 78/154] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:58.158226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.158514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.158796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.180454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 77/155] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:58.195811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:58.197533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:58.232359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.296392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 79/156] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:58.494507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.609036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.623738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26fb3a-013e-4e18-b45a-b38768c57983 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.667433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.668143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.668438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.668827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.682505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:58.687404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:58.688258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:58.689442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 78/157] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:58.694962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.722364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.917095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505b3a2-714f-4a1c-b198-e0679211fb39 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.938957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5002e-bb93-4f28-b26c-07e4ec54b1c9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.991635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.992276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.992557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.992833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.993520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:58.993828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:58.995241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.007529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 80/158] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:59.008821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.013330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.014180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.014593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.015113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.039202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.043935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:59.057643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.058554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:59.059355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 79/159] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:59.060191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.087292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.267631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41c4c-f9bf-4aa8-b69a-a37565493009 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.332413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb317-5624-4ff8-b462-695ac8929d50 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.363323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.364009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.364292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.364563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.393258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:59.400859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.401823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:44:59.402686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 81/160] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:59.404124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.413668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.414309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.414589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.414881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.416021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.418209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.420006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 80/161] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.420582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.600389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.623195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.655060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6a6e-3f06-47eb-86ec-910748b5b73d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.670130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10635b-6c48-47e4-b1aa-abc7b7ff9d2d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.742242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.743298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.743791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.744289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.745164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.745668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.747300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.749402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.750374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.750495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.750881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.757387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 82/162] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.760250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.775549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:59.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.781498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:44:59.782312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 81/163] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:59.783020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.891726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4981a4-ecf5-4e4f-b095-197f853b2ea4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.922556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.936328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.937050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.937329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.937599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.938268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.938572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.940173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.946193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e8ddf-863e-4211-b6cb-8ad60dde8178 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.959016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 83/164] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.960042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.991045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.991922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.992379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.992837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.996642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.076370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.078180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.209593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 82/165] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:00.210728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.217692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc8382-0702-4c56-9852-ad05753ac886 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.250822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.320990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3771c-9e34-40fe-a460-8712821354ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.321634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3771c-9e34-40fe-a460-8712821354ef None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.321948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3771c-9e34-40fe-a460-8712821354ef None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.322176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3771c-9e34-40fe-a460-8712821354ef None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.327435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4fc30-b314-4c03-8196-c632d80995b4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.327583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3771c-9e34-40fe-a460-8712821354ef None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.328997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3771c-9e34-40fe-a460-8712821354ef None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.331019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 84/166] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:00.331797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90253e-a444-4fb4-a96c-4b1a4fa2601f None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.332076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90253e-a444-4fb4-a96c-4b1a4fa2601f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.332349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90253e-a444-4fb4-a96c-4b1a4fa2601f None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.333553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 85/167] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:00.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90253e-a444-4fb4-a96c-4b1a4fa2601f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.368989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.369922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.370394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.370862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.371740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:00.372619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.374417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.385024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 83/168] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:00.386227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.500381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.504712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22066b69-8deb-4433-97de-1bc9564395f9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.550094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.551054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.551679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.552230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.553864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.554279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.556086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.558554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 84/169] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:00.559530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.680449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90253e-a444-4fb4-a96c-4b1a4fa2601f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.684052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d9475-8452-4f1b-86c1-1b3f1970d15d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.727699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.728340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.728630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.728905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.769310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.783564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15518d5a-3d01-4068-8d64-2877cadc6dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.784220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15518d5a-3d01-4068-8d64-2877cadc6dfe None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.784503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15518d5a-3d01-4068-8d64-2877cadc6dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.784771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15518d5a-3d01-4068-8d64-2877cadc6dfe None admin] PATH_INFO: `/v3/users/91cdfaf440314ad2b1f94da968a2f0eb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.791640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15518d5a-3d01-4068-8d64-2877cadc6dfe None admin] RBAC: Authorizing `identity:get_user(user_id=91cdfaf440314ad2b1f94da968a2f0eb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.793117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15518d5a-3d01-4068-8d64-2877cadc6dfe None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.794799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 86/170] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/users/91cdfaf440314ad2b1f94da968a2f0eb => generated 247 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:00.795986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.796264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.796532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.815404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.816896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.831831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:00.833026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:00.918586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 87/171] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:00.920459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.934201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 85/172] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:00.939554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.001877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.025790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.055941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34d8c9-7050-4435-bace-de758bbdfa7c None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.065217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f149af-b544-4d28-850c-51c8554df62e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.099109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.100306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.100669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.100949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] PATH_INFO: `/v3/users/0b672c7685154ab1a019a0a63150dcf1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.102078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] RBAC: Authorizing `identity:get_user(user_id=0b672c7685154ab1a019a0a63150dcf1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.103553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.103769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.104382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.104664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.104935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.109969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 88/173] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/users/0b672c7685154ab1a019a0a63150dcf1 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:01.110092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=0b672c7685154ab1a019a0a63150dcf1) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:01.110092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.110242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.110322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290c2af-ef18-4b8d-a03d-7a7053aa2480 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.114722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 86/174] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/role_assignments?user.id=0b672c7685154ab1a019a0a63150dcf1&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:01.123134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258417-86a2-47a6-9128-532835b3db60 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.123134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258417-86a2-47a6-9128-532835b3db60 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.123134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258417-86a2-47a6-9128-532835b3db60 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.123134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 87/175] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:01.123134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259187-7c6c-41fe-baba-b1e935dc61cf None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.123134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259187-7c6c-41fe-baba-b1e935dc61cf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.123620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259187-7c6c-41fe-baba-b1e935dc61cf None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.123620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 88/176] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:01.127189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259187-7c6c-41fe-baba-b1e935dc61cf None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.227401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8ee7a-91ba-4d58-922a-f1f49a818c4b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.254012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259187-7c6c-41fe-baba-b1e935dc61cf None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.259699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.270215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a37b1a-72f0-43bd-96bc-5a113ba23907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.270625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37b1a-72f0-43bd-96bc-5a113ba23907 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.271043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37b1a-72f0-43bd-96bc-5a113ba23907 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.271882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a37b1a-72f0-43bd-96bc-5a113ba23907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.285870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.293368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa046e-14d9-4fd4-b537-252964e9b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.294129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa046e-14d9-4fd4-b537-252964e9b6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.294417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa046e-14d9-4fd4-b537-252964e9b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.294693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa046e-14d9-4fd4-b537-252964e9b6a2 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.295888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa046e-14d9-4fd4-b537-252964e9b6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.297360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa046e-14d9-4fd4-b537-252964e9b6a2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.299053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 89/177] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:01.300199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649720b-6cba-4d1a-9fc0-7ecc3ae06893 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.300579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649720b-6cba-4d1a-9fc0-7ecc3ae06893 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.300859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649720b-6cba-4d1a-9fc0-7ecc3ae06893 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.302082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 90/178] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:01.303594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649720b-6cba-4d1a-9fc0-7ecc3ae06893 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.368364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37b1a-72f0-43bd-96bc-5a113ba23907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.369859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a37b1a-72f0-43bd-96bc-5a113ba23907 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.419796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649720b-6cba-4d1a-9fc0-7ecc3ae06893 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.455043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.455654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.455935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.456210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.473588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 89/179] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:01.475743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d12f0-f5dd-43b6-846c-666a0db34c7c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.476029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d12f0-f5dd-43b6-846c-666a0db34c7c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.476299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d12f0-f5dd-43b6-846c-666a0db34c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.495282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-0b8d12f0-f5dd-43b6-846c-666a0db34c7c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:01.496465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0b8d12f0-f5dd-43b6-846c-666a0db34c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:01.519569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.520028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.525017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.526491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.567306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 90/180] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:01.572148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.572438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.572711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.592418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:01.593624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:01.637671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 91/181] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:01.637671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.674910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 91/182] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:01.679896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.752286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.757036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34f94-df93-4330-a885-e73b3cdcbb64 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.787981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:01.803460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcece6-7140-4a66-a408-d45708c0c157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.804234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcece6-7140-4a66-a408-d45708c0c157 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.804520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcece6-7140-4a66-a408-d45708c0c157 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.804800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcece6-7140-4a66-a408-d45708c0c157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.805546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e1c2b-0c9b-4651-8415-ceeda5e0126d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.815836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcece6-7140-4a66-a408-d45708c0c157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1adcece6-7140-4a66-a408-d45708c0c157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:01.821532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:01.823006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 92/183] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:01.824296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.825668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.825668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.843957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:01.845167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:01.851960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.852580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.852862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.853135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.853866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] RBAC: Adding query filter params (role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=0b672c7685154ab1a019a0a63150dcf1) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:01.854176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.855929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.862089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 92/184] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/role_assignments?user.id=0b672c7685154ab1a019a0a63150dcf1&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=238ba40fcefe4923bb27057b1074fbef => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:01.862869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:01.937349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 93/185] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:01.938403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.013900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a33a1-9706-4756-9aa2-c22f08094e26 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.051965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.052671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.052913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.053351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.053351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.054620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.056793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.058029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 93/186] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.059385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.068960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e55eaa-dc8a-4984-95bd-4fe5a293896c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.084261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.111895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.112545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.112831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.113107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] PATH_INFO: `/v3/projects/2a052524645a43918a55bcd8e2546b6a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.115422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] RBAC: Authorizing `identity:get_project(project_id=2a052524645a43918a55bcd8e2546b6a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.117561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.119285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 94/187] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:01 2022] GET /identity/v3/projects/2a052524645a43918a55bcd8e2546b6a => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.124601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.186045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfaf9-ddfc-431c-87d4-27489122f9a2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.225307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.225924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.226209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.226490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.227611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:02.227926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.229611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.238866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 94/188] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.238998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985da88-edda-4911-ba64-9bc760f29734 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.239238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.273511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.274132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.274410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.274683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.284607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:02.290639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:02.291636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:02.292422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 95/189] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:02.293206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.311478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.325382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.344916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43cf13-876c-435b-9d65-e594cf37b3f0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.377427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.378061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.378343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.378615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.389358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:02.393135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25a601-e1a8-425e-a53e-51b50c1ab466 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:02.395270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:02.396367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 95/190] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:02.397236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.428997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.429599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.429881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.430154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.430908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=91cdfaf440314ad2b1f94da968a2f0eb) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:02.431222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.432670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.438319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 96/191] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/role_assignments?user.id=91cdfaf440314ad2b1f94da968a2f0eb&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.439126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.518340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786dc21-4f20-4376-b06f-a7a4cc1250db None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.535406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.559222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d51a-31c4-4d82-95b4-8cbe031f09dc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.561079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.561700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.561981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.562253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.563031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=2a052524645a43918a55bcd8e2546b6a, user.id=0b672c7685154ab1a019a0a63150dcf1) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:02.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.564787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.572769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 96/192] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/role_assignments?user.id=0b672c7685154ab1a019a0a63150dcf1&scope.project.id=2a052524645a43918a55bcd8e2546b6a&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.573641 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.587633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.622967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.622967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.622967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.623290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.625337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.627390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.629331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 97/193] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.632409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.723581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2ab92-969b-46e2-929e-d82f1d06bd37 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.763095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.763744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.764086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.764467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] PATH_INFO: `/v3/users/4c40b2c706704ffbbe293b53a31444b5` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.780453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] RBAC: Authorizing `identity:get_user(user_id=4c40b2c706704ffbbe293b53a31444b5)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.781918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.783833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 97/194] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/users/4c40b2c706704ffbbe293b53a31444b5 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.783942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02994fdd-6785-455e-831b-1a1487af2292 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.784538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.834187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.834807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.835115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.835383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.836048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:02.836383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.837784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.848872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 98/195] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.850766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.858855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.864492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:02.898059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f86e21-1e21-4652-88b9-5420745d5942 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.933043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.933653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.933933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.934203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.934905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:02.935211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.936570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:02.945599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 98/196] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.946410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.956696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1aac08-9ffa-4f64-af9e-36ceb9c5d0ce None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:02.995041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:02.995646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.995921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.996191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.996868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:02.998358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.021236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 99/197] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:02 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:03.022068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.054769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94adee-eaab-4819-b4d6-12bf4e25d2d8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.085605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.098384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.099895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.100165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.100674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.112245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.152947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383190f2-32c0-4965-8b0a-ddbc0645c745 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.189063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.191496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.191819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.192579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.193635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.193635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] PATH_INFO: `/v3/users/4c40b2c706704ffbbe293b53a31444b5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.196059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] RBAC: Authorizing `identity:get_user(user_id=4c40b2c706704ffbbe293b53a31444b5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.198716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.205711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 100/198] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/users/4c40b2c706704ffbbe293b53a31444b5 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:03.205711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.323433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a018c3-812c-411b-a540-fdfb7bce703b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.329183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 99/199] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:03.329183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.362559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.374665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.375404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.375687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.375962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] PATH_INFO: `/v3/users/2586d38983cf442a957e3ca9f03a6c3b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.378149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] RBAC: Authorizing `identity:get_user(user_id=2586d38983cf442a957e3ca9f03a6c3b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.380029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.382777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 101/200] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/users/2586d38983cf442a957e3ca9f03a6c3b => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:03.383678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.391677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.456000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae102-c3ea-40d9-bf97-dab4a49b93c1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.503899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.504520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.504797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.505070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.518942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadd679-f10f-48ed-978f-90eaabc2e384 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.558365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.559042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.559328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.559606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.560742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.562219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.564315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 102/201] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:03.564934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.582809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.584570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.612579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.640833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.678298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddb18c-49ae-42d5-aba1-afbc9d08c289 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.704404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 100/202] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:03.705333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.716005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.716835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.717281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.717700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.790683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.792428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.813653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe925ae-8d77-4d4a-9dae-745edf1df988 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.844908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.854876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.855564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.855768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.856102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.856974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=4c40b2c706704ffbbe293b53a31444b5) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:03.857138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.858569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.864318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 101/203] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/role_assignments?user.id=4c40b2c706704ffbbe293b53a31444b5&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:03.865333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.868404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.911057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 103/204] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:03.913458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.982027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf260c-7bba-46cd-b6ed-7c273a149b45 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.019185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.019849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.020136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.020448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.022206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.023882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.030691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 102/205] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:04.032064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.036090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf854d-9654-4285-8423-b2f11d6437ce None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.076698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.077732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.078215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.078712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.080158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.081873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.084918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 104/206] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:04.091121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.103797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.135959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.147875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234d840-24b5-4191-8151-ebb27afb5f9e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.182864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.183488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.183769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.184046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.184971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] RBAC: Adding query filter params (role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=4c40b2c706704ffbbe293b53a31444b5) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:04.185088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.186513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057148f-7eb7-45f0-9912-b11291c9f774 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.198466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 103/207] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/role_assignments?user.id=4c40b2c706704ffbbe293b53a31444b5&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=238ba40fcefe4923bb27057b1074fbef => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:04.205803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb1998-27f5-4435-a7a0-01e827cf49f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.249785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.250766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.251278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.251756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.252744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] RBAC: Adding query filter params (role.id=b7d97d6c79bf4e0c90bce6aec6777332, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055, user.id=2586d38983cf442a957e3ca9f03a6c3b) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:04.253279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.255022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ea6e7-0649-4e73-8aa8-4e41ed49b7cd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.267056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 105/208] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/role_assignments?user.id=2586d38983cf442a957e3ca9f03a6c3b&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=b7d97d6c79bf4e0c90bce6aec6777332 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:05.554540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947a930-32e7-4178-ad46-f7b33d49b46e None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.555136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947a930-32e7-4178-ad46-f7b33d49b46e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.555213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947a930-32e7-4178-ad46-f7b33d49b46e None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.556934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 104/209] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:05.563152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8e9b-0f6f-47b3-b312-6e8e788622e3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.566783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8e9b-0f6f-47b3-b312-6e8e788622e3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.566783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a8e9b-0f6f-47b3-b312-6e8e788622e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.584742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-136a8e9b-0f6f-47b3-b312-6e8e788622e3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:05.585944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-136a8e9b-0f6f-47b3-b312-6e8e788622e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:05.657595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7fe60-4323-44a9-a182-257135b4ba82 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.657935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7fe60-4323-44a9-a182-257135b4ba82 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.658158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7fe60-4323-44a9-a182-257135b4ba82 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.659042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 106/210] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:05.659717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 105/211] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:05.668686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.668970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.669245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.688305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:05.689487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:05.717575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.767072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 107/212] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:05.933361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7fe60-4323-44a9-a182-257135b4ba82 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.046027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7fe60-4323-44a9-a182-257135b4ba82 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.070311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.071444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.090021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.091127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.091658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.092111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.104719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:06.110364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:06.111271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:06.112060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:06.114371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 106/213] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:06.120777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.187702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b734c72-54c7-4dbb-b03c-a8b81a15e1a0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.225199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.225857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.226139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.226412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.227760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.229355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.231199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 108/214] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:06.237207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.247574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c453-6b17-41ae-b616-c670ca73bc50 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.284442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.293148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.293826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.294115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.294394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.295097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:06.295459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.296858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.308229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 107/215] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:06.314414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.363165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.368463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5871fd-3b7a-4177-a32b-cca577d63d40 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.406321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9fbc1-0480-4c18-a63e-c76bac7e4baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.406953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9fbc1-0480-4c18-a63e-c76bac7e4baf None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.407232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9fbc1-0480-4c18-a63e-c76bac7e4baf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.407507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9fbc1-0480-4c18-a63e-c76bac7e4baf None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.408157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9fbc1-0480-4c18-a63e-c76bac7e4baf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.409604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9fbc1-0480-4c18-a63e-c76bac7e4baf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.424975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216c33-87d5-4153-ab00-ebdda16754cb None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.434005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 109/216] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:06 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:06.435260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8568-76d7-4c60-9b08-921a9ff20aab None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.435553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8568-76d7-4c60-9b08-921a9ff20aab None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.435832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8568-76d7-4c60-9b08-921a9ff20aab None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.441721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 110/217] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:06.444911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369d960-94ec-42e5-a39a-ebfe4af1d666 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.445219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369d960-94ec-42e5-a39a-ebfe4af1d666 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.445491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369d960-94ec-42e5-a39a-ebfe4af1d666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.470947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d369d960-94ec-42e5-a39a-ebfe4af1d666 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:06.472432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d369d960-94ec-42e5-a39a-ebfe4af1d666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:06.479808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.480809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.481366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.482555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] PATH_INFO: `/v3/users/2f02064632ea4305bedd72d75cd91dfe` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.504183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] RBAC: Authorizing `identity:get_user(user_id=2f02064632ea4305bedd72d75cd91dfe)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.508390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.512021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 108/218] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/users/2f02064632ea4305bedd72d75cd91dfe => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:06.517090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.566859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 111/219] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:06.566859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ee055-6e3f-4eba-a518-83f8fb50b358 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.566859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ee055-6e3f-4eba-a518-83f8fb50b358 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.567537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ee055-6e3f-4eba-a518-83f8fb50b358 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.570584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 112/220] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:06.576803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ac72-1529-46d1-829a-401d22305060 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.577246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ac72-1529-46d1-829a-401d22305060 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.577595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ac72-1529-46d1-829a-401d22305060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.605367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6974ac72-1529-46d1-829a-401d22305060 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:06.606596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6974ac72-1529-46d1-829a-401d22305060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:06.622299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.664351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73593ac6-5671-4218-838c-5e716f76779c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.665907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.697454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 113/221] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:06.708585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.709213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.709565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.709909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.790430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.791938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.851220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ac72-1529-46d1-829a-401d22305060 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.923584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 109/222] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:45:06.930655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.932680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.995522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ac72-1529-46d1-829a-401d22305060 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.039323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.039969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.040248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.040517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.053306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:07.058221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:07.059141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:07.059923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 114/223] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:07.060562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.063959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18028538-0608-4d78-8fe1-66d59b19802c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.070437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.109154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.110188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.110616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.111054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.112338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.114828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.117639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 110/224] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:07.118600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.171479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef9aa8-2a06-4d47-bfda-8b44277e2777 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.208252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.209020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.209776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.210071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.210402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.222137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:07.228467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:07.229576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:07.230562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:07.232589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 115/225] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:07.233481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.256939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc9787-6b79-4925-aaca-bfbc2c829047 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.303005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.303898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.304316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.304731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.305542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.305991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.307074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.309324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.320495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 111/226] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.321639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.346422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86412ce9-c3ef-43e7-86ce-80652b7b5434 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.382620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.383625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.383745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.385124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.385962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=2f02064632ea4305bedd72d75cd91dfe) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.386365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.388076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.394383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 116/227] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/role_assignments?user.id=2f02064632ea4305bedd72d75cd91dfe&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.395304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.461161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680ac23-d851-4015-bd19-d71f6a1adb2f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.532068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.540172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.540805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.541086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.541357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.542038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.542345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.543768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.561816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 112/228] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.568664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.590693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d22228-4a08-48e0-a560-108c9d94d216 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.636848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.647050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.647883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.648168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.648878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] PATH_INFO: `/v3/users/4c40b2c706704ffbbe293b53a31444b5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.649793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] RBAC: Authorizing `identity:get_user(user_id=4c40b2c706704ffbbe293b53a31444b5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.651619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.653758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 117/229] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/users/4c40b2c706704ffbbe293b53a31444b5 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.659672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.721963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3472d-470d-4c2d-bf41-e874e9bd4ff6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.762496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.764201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.765307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.765789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] PATH_INFO: `/v3/users/4c40b2c706704ffbbe293b53a31444b5` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.768444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] RBAC: Authorizing `identity:get_user(user_id=4c40b2c706704ffbbe293b53a31444b5)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.775234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.777349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 113/230] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/users/4c40b2c706704ffbbe293b53a31444b5 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.788034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.842576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa140d-2c37-42ef-bfcb-9ca2adc9de23 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.873766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.917137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08598e-08c6-404b-aa94-55857742107b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.920340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.920853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.921144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.921422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.928678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.930162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.931952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 118/231] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.938460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.957780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.958804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.959305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.959763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.961149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.962776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.964957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 114/232] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.971216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:08.067313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:08.132118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76b3ba-5b86-4e9e-b2f4-e977cd27c92e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.155759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130c324-4fba-4f87-b8ac-890c8530b0dc None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.195317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4780e-f8c1-474b-8d08-353b528888ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.195989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4780e-f8c1-474b-8d08-353b528888ac None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.196272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4780e-f8c1-474b-8d08-353b528888ac None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.197757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4780e-f8c1-474b-8d08-353b528888ac None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/4c40b2c706704ffbbe293b53a31444b5/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.200433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4780e-f8c1-474b-8d08-353b528888ac None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8ffdfac9ab4b53a043e4aa806e5446, user_id=4c40b2c706704ffbbe293b53a31444b5, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.201889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4780e-f8c1-474b-8d08-353b528888ac None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.211912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:08.222705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 119/233] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:07 2022] PUT /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/4c40b2c706704ffbbe293b53a31444b5/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:45:08.227580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebc20c-ec32-47b9-b38e-b70497274a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.228446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebc20c-ec32-47b9-b38e-b70497274a58 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.228731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebc20c-ec32-47b9-b38e-b70497274a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.229010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebc20c-ec32-47b9-b38e-b70497274a58 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/4c40b2c706704ffbbe293b53a31444b5/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.232008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebc20c-ec32-47b9-b38e-b70497274a58 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8ffdfac9ab4b53a043e4aa806e5446, user_id=4c40b2c706704ffbbe293b53a31444b5, role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.233647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebc20c-ec32-47b9-b38e-b70497274a58 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.251290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 115/234] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:07 2022] PUT /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/4c40b2c706704ffbbe293b53a31444b5/roles/238ba40fcefe4923bb27057b1074fbef => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:09.246587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a2c2-588e-4a64-ad63-f209a380df15 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.247098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a2c2-588e-4a64-ad63-f209a380df15 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.247363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7a2c2-588e-4a64-ad63-f209a380df15 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.249790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 120/235] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:09.256054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.256602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.257041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.277832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb1a81-a653-45f2-8d64-ae38791ccd33 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.278136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb1a81-a653-45f2-8d64-ae38791ccd33 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.278390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb1a81-a653-45f2-8d64-ae38791ccd33 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.280873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 121/236] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:09.286293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.286566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.287442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.289970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:09.291247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:09.308964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:09.310152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:09.417980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 122/237] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:09.438918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 116/238] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:09.780917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:09.840338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:09.858934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:09.900801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3da4e-cee0-4837-bc30-03302a7b450a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:09.972923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:09.973580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.973864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.974738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.976018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:09.982426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:09.984163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 123/239] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:09.997821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.070125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fb02f-9f75-4ecb-9d7b-12317b12013e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.097383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.105669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e84c1f-3280-4add-b5b7-81c0efbc49f5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.116768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1b489-603d-4cd0-bdc0-46ddffd934c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.117788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1b489-603d-4cd0-bdc0-46ddffd934c1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.118265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1b489-603d-4cd0-bdc0-46ddffd934c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.118734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1b489-603d-4cd0-bdc0-46ddffd934c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.130928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.132909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a1b489-603d-4cd0-bdc0-46ddffd934c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-e6a1b489-603d-4cd0-bdc0-46ddffd934c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:10.137895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:10.138772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:10.139598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 117/240] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:10.140520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bd16b-33f3-4181-b359-1b5c34dd3d00 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.140965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bd16b-33f3-4181-b359-1b5c34dd3d00 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.141368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bd16b-33f3-4181-b359-1b5c34dd3d00 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.142861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 118/241] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:10.145701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bd16b-33f3-4181-b359-1b5c34dd3d00 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.158247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cb492-53e3-4aaf-b03b-0b5207c6184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.159030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb492-53e3-4aaf-b03b-0b5207c6184a None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.159411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb492-53e3-4aaf-b03b-0b5207c6184a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.159781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cb492-53e3-4aaf-b03b-0b5207c6184a None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.160664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb492-53e3-4aaf-b03b-0b5207c6184a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.162871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cb492-53e3-4aaf-b03b-0b5207c6184a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.186993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 124/242] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:09 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:10.188324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.188856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.189327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.222050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:10.226374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:10.320090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bd16b-33f3-4181-b359-1b5c34dd3d00 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.364386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.366131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.366729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.367239 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.368140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:10.368644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.370281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.380096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 119/243] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:10.387057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.409054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 125/244] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:10.438372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.501398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674da2fd-f682-4d70-81b5-6017b2b2e657 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.538773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.539736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.540167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.540657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] PATH_INFO: `/v3/users/74503ee96439434faad6a961ce6ddf91` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.555698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] RBAC: Authorizing `identity:get_user(user_id=74503ee96439434faad6a961ce6ddf91)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.557513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.559560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 120/245] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/users/74503ee96439434faad6a961ce6ddf91 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:10.565426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.692709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2d2a-704f-4199-830b-0340ef480201 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.707689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.711415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.761611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2648d-c604-4b83-a080-cc9dda5c3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.762237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2648d-c604-4b83-a080-cc9dda5c3ded None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.762513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2648d-c604-4b83-a080-cc9dda5c3ded None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.762781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2648d-c604-4b83-a080-cc9dda5c3ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.843566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2648d-c604-4b83-a080-cc9dda5c3ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.844434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2648d-c604-4b83-a080-cc9dda5c3ded None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.871360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.875556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584620d7-aa31-45a7-bab0-d6dedfd1a82c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.917481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a3c4-6014-4d3b-bc97-15848f387536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.918481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a3c4-6014-4d3b-bc97-15848f387536 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.919082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a3c4-6014-4d3b-bc97-15848f387536 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.919576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a3c4-6014-4d3b-bc97-15848f387536 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.931331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355a3c4-6014-4d3b-bc97-15848f387536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-2355a3c4-6014-4d3b-bc97-15848f387536 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:10.936613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:10.937507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:10.938362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:10.940664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 121/246] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:10.942110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f875ed-7695-47bc-8546-18015d04d521 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.942604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f875ed-7695-47bc-8546-18015d04d521 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.943802 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f875ed-7695-47bc-8546-18015d04d521 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.945519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 122/247] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:10.950981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f875ed-7695-47bc-8546-18015d04d521 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.961000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 126/248] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:10.962508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.962829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.963130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.984438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:10.985752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:10.988862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.074477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 127/249] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:11.075251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.097953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f875ed-7695-47bc-8546-18015d04d521 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.139134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.139750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.140031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.140296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.141294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:11.141604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.143060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.159527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 123/250] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.163491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.206800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb221a9-c72e-4161-a6f8-83aacf864f82 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.244422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.245470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.245940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.246399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.247767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.249418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.254956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 128/251] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.258471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.269771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.282162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4c58f-4cbc-4a41-869d-6ea19b05767f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.321547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.322333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.322530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.322805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] PATH_INFO: `/v3/users/2f02064632ea4305bedd72d75cd91dfe` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.324465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] RBAC: Authorizing `identity:get_user(user_id=2f02064632ea4305bedd72d75cd91dfe)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.325995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.327917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 124/252] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/users/2f02064632ea4305bedd72d75cd91dfe => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.334355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.441600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.446572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0ab82-0431-40bf-bcc9-0bbf9f1faca0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.455907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8178ed-5e8f-4ffd-bd26-dd9ce78e74f2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.494392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32045ff3-bd74-4d16-b9ce-6196a2dae4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.495418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32045ff3-bd74-4d16-b9ce-6196a2dae4fc None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.495794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32045ff3-bd74-4d16-b9ce-6196a2dae4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.496071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32045ff3-bd74-4d16-b9ce-6196a2dae4fc None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.497567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32045ff3-bd74-4d16-b9ce-6196a2dae4fc None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.499088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32045ff3-bd74-4d16-b9ce-6196a2dae4fc None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.500747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 125/253] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.501885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb286e8b-6c25-4d0b-be6f-ec6f2e413903 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.502167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb286e8b-6c25-4d0b-be6f-ec6f2e413903 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.502441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb286e8b-6c25-4d0b-be6f-ec6f2e413903 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.503933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 126/254] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:11.504643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb286e8b-6c25-4d0b-be6f-ec6f2e413903 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.522189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.523281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.523826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.524101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.525103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] RBAC: Adding query filter params (role.id=b7d97d6c79bf4e0c90bce6aec6777332, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=74503ee96439434faad6a961ce6ddf91) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:11.525416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.527299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.535207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 129/255] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/role_assignments?user.id=74503ee96439434faad6a961ce6ddf91&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=b7d97d6c79bf4e0c90bce6aec6777332 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:11.536069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.640766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.680822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5dd67-2928-4370-9bc5-27d8a6452501 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.701625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb286e8b-6c25-4d0b-be6f-ec6f2e413903 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.721602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef60316-4a11-4079-82e2-b2d07e508551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.722373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef60316-4a11-4079-82e2-b2d07e508551 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.722717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef60316-4a11-4079-82e2-b2d07e508551 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.723020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef60316-4a11-4079-82e2-b2d07e508551 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/2f02064632ea4305bedd72d75cd91dfe/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.725847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef60316-4a11-4079-82e2-b2d07e508551 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8ffdfac9ab4b53a043e4aa806e5446, user_id=2f02064632ea4305bedd72d75cd91dfe, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.727487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef60316-4a11-4079-82e2-b2d07e508551 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.746879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.747549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.747833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.748110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.749144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 130/256] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:11 2022] PUT /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/2f02064632ea4305bedd72d75cd91dfe/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:11.755274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.757265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.757555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.777019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:11.788676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.789626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:11.791857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 127/257] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.795852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.797080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:11.798314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:11.834374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.947995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 131/258] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:11.972647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9183bd-ca79-437f-8721-17d132f65bdd None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.012708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.013348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.013722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.014065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.014830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.015235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.016746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.026940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 128/259] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.032739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.114443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.157211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1387e-4b7a-4b48-a2e0-4f8c82a074f2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.192448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.193822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.194213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.194753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] PATH_INFO: `/v3/users/4c40b2c706704ffbbe293b53a31444b5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.195927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] RBAC: Authorizing `identity:get_user(user_id=4c40b2c706704ffbbe293b53a31444b5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.197477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.199227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 132/260] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/users/4c40b2c706704ffbbe293b53a31444b5 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.204781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.317493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.339635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ff67f-ae77-4508-b91b-e34f09c471a9 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.385957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.386581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.386877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.387289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.402579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.508333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.510304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.596637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dad3a-968d-4246-b069-4b34feb4ef94 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.646731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.654106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.654753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.655046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.655321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.680646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:12.685865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:12.686821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:12.687636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:12.688784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 133/261] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:12.689211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.697230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.727974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 129/262] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:12.734198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.910098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e6de-a048-411c-87a7-3d53e0c8f0f4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.961804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467693ce-fa7e-4136-b15e-31e380ef655a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.998022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.998684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.998999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.999274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.000448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.002826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.004809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 130/263] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:13.011574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.040457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.041071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.041349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.041618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.042343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:13.042646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.044054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.062048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 134/264] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:13.068931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.135012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.169773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.245882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92159258-e8b7-4c45-9d49-d6bd0deb2673 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.284992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef180e-0912-441b-b62a-2286d45d7a59 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.295458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.296353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.296876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.297366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.298300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=4c40b2c706704ffbbe293b53a31444b5) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:13.298812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.301033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.314206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 131/265] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/role_assignments?user.id=4c40b2c706704ffbbe293b53a31444b5&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.370507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.371243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.371528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.371813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] PATH_INFO: `/v3/users/4c40b2c706704ffbbe293b53a31444b5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.372983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] RBAC: Authorizing `identity:get_user(user_id=4c40b2c706704ffbbe293b53a31444b5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.374564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.376508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 135/266] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/users/4c40b2c706704ffbbe293b53a31444b5 => generated 283 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.382675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.500546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d429491-4786-4a43-93d9-d1045be8cc80 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.515932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.546270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332059bb-6a49-4c63-ad10-957fb23d2feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.547280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332059bb-6a49-4c63-ad10-957fb23d2feb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.547791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332059bb-6a49-4c63-ad10-957fb23d2feb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.548292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332059bb-6a49-4c63-ad10-957fb23d2feb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.643735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332059bb-6a49-4c63-ad10-957fb23d2feb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.645591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332059bb-6a49-4c63-ad10-957fb23d2feb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.768722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 132/267] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:13.775069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.924691 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cf394-7794-424f-9141-7c4cd619d7cf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.947009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.960153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743565c-3332-4a80-8cfa-22ee48e315b0 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.961133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743565c-3332-4a80-8cfa-22ee48e315b0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.961993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743565c-3332-4a80-8cfa-22ee48e315b0 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.964290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 133/268] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:13.969065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.969723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.970012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.970291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.970910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77688-27c8-4349-981d-a4d2778b11c9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.971401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77688-27c8-4349-981d-a4d2778b11c9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.971886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77688-27c8-4349-981d-a4d2778b11c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.975696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.982861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.982861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 136/269] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.987355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.012308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9cb77688-27c8-4349-981d-a4d2778b11c9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:14.013864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9cb77688-27c8-4349-981d-a4d2778b11c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:14.027647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.113039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8aece-ab52-4d4e-9c4d-40bed772994f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.122817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 134/270] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:14.130888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6657a8-71b4-4aa8-9220-bf43eefe26c9 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.130888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6657a8-71b4-4aa8-9220-bf43eefe26c9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.130888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6657a8-71b4-4aa8-9220-bf43eefe26c9 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.130888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 135/271] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:14.137626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.137909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.138510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.160335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:14.162654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:14.211565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.212354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.212862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.213345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.214277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] RBAC: Adding query filter params (role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=4c40b2c706704ffbbe293b53a31444b5) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:14.214801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.216481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.233351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 137/272] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/role_assignments?user.id=4c40b2c706704ffbbe293b53a31444b5&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=238ba40fcefe4923bb27057b1074fbef => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:14.240501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 136/273] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:14.536151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.571969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.603797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.654372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.666394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca1c6-be04-431f-820c-dff58ad3eaae None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.689146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64dba7-c196-427f-a330-2e8ae6ab6a88 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.709206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.710257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.710741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.711319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.712828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.714576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.716826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 138/274] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:14.722028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.726595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.727340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.727599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.727898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.747984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:14.753161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:14.754123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:14.754971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 137/275] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:14.763164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.864374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396b476-8e30-4f24-b986-9f1ace09dd48 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.890508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.910804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fe301-1e84-4933-a876-fe65a83de6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.912070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe301-1e84-4933-a876-fe65a83de6be None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.912399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe301-1e84-4933-a876-fe65a83de6be None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.912687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fe301-1e84-4933-a876-fe65a83de6be None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.913415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fe301-1e84-4933-a876-fe65a83de6be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.914936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fe301-1e84-4933-a876-fe65a83de6be None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.916022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ed59d-6358-416d-8372-9f86387e245f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.940004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 139/276] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:14 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:14.941528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e463-bc51-4ec8-97fb-d2a00ce67c61 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.941883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e463-bc51-4ec8-97fb-d2a00ce67c61 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.942230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e463-bc51-4ec8-97fb-d2a00ce67c61 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.944923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 140/277] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:14.950464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c18e9-10e7-4103-a819-afe99ddab83c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.950803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c18e9-10e7-4103-a819-afe99ddab83c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.951202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c18e9-10e7-4103-a819-afe99ddab83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.959921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.971276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.972016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.972373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.972724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.973495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:14.974471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.976056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.986113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 138/278] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:14.991561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-142c18e9-10e7-4103-a819-afe99ddab83c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:14.992503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.993461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-142c18e9-10e7-4103-a819-afe99ddab83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:15.078023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 141/279] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:15.079673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ef0be-9b01-4312-b801-61ecca9759c0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.080140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ef0be-9b01-4312-b801-61ecca9759c0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.080588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ef0be-9b01-4312-b801-61ecca9759c0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.082103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 142/280] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:15.092676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.093022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.093224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.114258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:15.115916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:15.117586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851c65a-e99e-416b-8322-a7cccf0afeab None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:15.166231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:15.166904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.167577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.167856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] PATH_INFO: `/v3/users/155802b7e89c44c6881dd01666ec331e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.185506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] RBAC: Authorizing `identity:get_user(user_id=155802b7e89c44c6881dd01666ec331e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:15.187031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:15.188736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 139/281] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:14 2022] GET /identity/v3/users/155802b7e89c44c6881dd01666ec331e => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:15.195847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.196810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 143/282] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:15.256523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:15.308227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77000409-1cf9-467f-89d6-b883db693f11 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:15.346664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:15.347315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.347596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.347880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.348612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, user.id=155802b7e89c44c6881dd01666ec331e, scope.system=all) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:15.348979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:15.350609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:15.356178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 140/283] 10.222.0.72 () {56 vars in 1308 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/role_assignments?user.id=155802b7e89c44c6881dd01666ec331e&scope.system=all&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:15.358693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.407909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:15.489101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d9557-bc50-49d7-97fe-098fe588cdab None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:15.490450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.539578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:15.540601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.541119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.541592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.555347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:15.563338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:15.564480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:15.565264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:15.568457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 144/284] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:15.577918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.637731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca725641-ff83-4898-96bc-52b84f497f5c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:15.664454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:15.686042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:15.687086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.687701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.688174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.701122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:15.706392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:15.707321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:15.708123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:15.716111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 141/285] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:15.717589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.795860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:15.809692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085b528-08e8-47be-8d73-ba0eb6729b5c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:15.828595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f79f78-18f7-4af3-bce7-68a92f50e9f8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:15.865745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:15.866732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.867247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.867721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.868591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:15.869118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:15.870722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:15.885468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 142/286] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:15.887056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.931688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:15.932508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.932508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.932742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.933422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:15.933840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:15.935163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:15.964314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 145/287] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:15.972659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:15.982741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.039972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b2660-4bdb-412f-8727-acd0b1a1bd35 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.081831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.082853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.083954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.084511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] PATH_INFO: `/v3/users/74503ee96439434faad6a961ce6ddf91` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.085898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] RBAC: Authorizing `identity:get_user(user_id=74503ee96439434faad6a961ce6ddf91)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.087662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.094572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 143/288] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/users/74503ee96439434faad6a961ce6ddf91 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:16.095986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.203483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc259a49-3a8e-4b47-a461-c52177ac0c84 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.213698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef975efa-0580-4671-8fd6-fd2e6ea190ed None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.241402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.249309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.250359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.250896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.251529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.254561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.257507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.267084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 144/289] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:16.268090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.271838 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.272825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.273469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.273947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] PATH_INFO: `/v3/users/2f02064632ea4305bedd72d75cd91dfe` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.275363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] RBAC: Authorizing `identity:get_user(user_id=2f02064632ea4305bedd72d75cd91dfe)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.277134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.279412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 146/290] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:15 2022] GET /identity/v3/users/2f02064632ea4305bedd72d75cd91dfe => generated 297 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:16.295832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.388434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.391783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2095dcd-2d16-42e8-b0ad-eee497d50078 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.434406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.435099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.435381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.435658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/74503ee96439434faad6a961ce6ddf91/roles/b7d97d6c79bf4e0c90bce6aec6777332` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.439399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8ffdfac9ab4b53a043e4aa806e5446, user_id=74503ee96439434faad6a961ce6ddf91, role_id=b7d97d6c79bf4e0c90bce6aec6777332)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.442114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.468181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 145/291] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:16 2022] PUT /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/users/74503ee96439434faad6a961ce6ddf91/roles/b7d97d6c79bf4e0c90bce6aec6777332 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:16.533728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242964b6-3216-4e8c-8a0b-2137456592f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.577504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.578125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.578455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.578735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.673226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.674686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.704709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.785115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 147/292] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:16.790692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.837959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.929247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00547336-02df-448c-b1d8-7e4d7be33776 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.969064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd55716-1b42-4fe7-80a3-49eb6c303992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.969707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd55716-1b42-4fe7-80a3-49eb6c303992 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.969989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd55716-1b42-4fe7-80a3-49eb6c303992 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.970261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd55716-1b42-4fe7-80a3-49eb6c303992 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.975608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd55716-1b42-4fe7-80a3-49eb6c303992 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.977490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd55716-1b42-4fe7-80a3-49eb6c303992 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.979379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 146/293] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:16.985378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.115286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a06db-e0b4-4c02-8925-e56fadb296f6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.148102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.160070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.160684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.160968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.161270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.162636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=2f02064632ea4305bedd72d75cd91dfe) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:17.162974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.164468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce1598-cc64-41ed-aada-c85ed75ae95f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.171349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 148/294] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/role_assignments?user.id=2f02064632ea4305bedd72d75cd91dfe&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:17.910662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484dc4-8c74-42bc-995d-15dfebc14212 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.911201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484dc4-8c74-42bc-995d-15dfebc14212 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.911325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484dc4-8c74-42bc-995d-15dfebc14212 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.913033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 147/295] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:17.918540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5a2bf-5edb-45b4-b1f2-7d3ad3497604 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.918854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5a2bf-5edb-45b4-b1f2-7d3ad3497604 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.919189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5a2bf-5edb-45b4-b1f2-7d3ad3497604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.944080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-93c5a2bf-5edb-45b4-b1f2-7d3ad3497604 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:17.945532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-93c5a2bf-5edb-45b4-b1f2-7d3ad3497604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:17.969659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0cfe-fb7d-491e-bd0d-f08d944c737a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.969958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0cfe-fb7d-491e-bd0d-f08d944c737a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.970223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f0cfe-fb7d-491e-bd0d-f08d944c737a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.971606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 148/296] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:17.979409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.979687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.979968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.005067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:18.005067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:18.029107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 149/297] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:18.040556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff61e1-3938-4625-93d8-686b680512a8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.040846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff61e1-3938-4625-93d8-686b680512a8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.041119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff61e1-3938-4625-93d8-686b680512a8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.042583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 150/298] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:18.048638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.048909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.049193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.069970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:18.071694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:18.091787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 149/299] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:18.144543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 151/300] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:18.302527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.334351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.380671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.407914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e0ba1-287b-4eb4-856a-6ee8cc246697 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.434916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.445289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.446106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.446733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.447065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.460680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:18.465984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:18.466868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:18.467680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 150/301] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:18.468323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.491158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4cec1-c2ac-4a7f-a386-68a25095b3ad None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.526306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.526979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.527258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.527531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.528625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.530077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.531851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 152/302] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:18.532433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.578878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.590971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f26e5-44e2-4ba4-9a8e-aac420757a4c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.628985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.629690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.630609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.630972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.639949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535b52c-ed5c-455c-8d8e-0e47ea6f0ab7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.641444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:18.646522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:18.647418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:18.648256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 151/303] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:18.648918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.660454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.679084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.679696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.679974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.680245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.680903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.681206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.682577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.691032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 153/304] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.691856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.763071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc269233-9f5e-4a5e-ba15-b31246239acc None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.801847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.802468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.802980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.803262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] PATH_INFO: `/v3/groups` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.803898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.804016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc1eac-aaec-4d6c-a48d-509a812d2710 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.805483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.826742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 152/305] 10.222.0.72 () {60 vars in 1134 bytes} [Mon Sep 26 17:45:18 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:18.828238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.847843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.851913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.853372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.853852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.854264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.855225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.855645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.857887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.869463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 154/306] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.875108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.925154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.971360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2f689-c7b9-45ce-965b-2e35c67ca1b7 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.002430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa684810-d5b1-402a-80d9-17c64a069539 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.020531 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bffba-0475-470c-b409-74d952134909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.022146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bffba-0475-470c-b409-74d952134909 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.022605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bffba-0475-470c-b409-74d952134909 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.023117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bffba-0475-470c-b409-74d952134909 None admin] PATH_INFO: `/v3/users/2e70eae957dd4b36ad4d6eba22465d16` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.040796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bffba-0475-470c-b409-74d952134909 None admin] RBAC: Authorizing `identity:get_user(user_id=2e70eae957dd4b36ad4d6eba22465d16)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.042700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bffba-0475-470c-b409-74d952134909 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.044246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.044826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 153/307] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/users/2e70eae957dd4b36ad4d6eba22465d16 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:19.044937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.045154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.045432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] PATH_INFO: `/v3/users/155802b7e89c44c6881dd01666ec331e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.046613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] RBAC: Authorizing `identity:get_user(user_id=155802b7e89c44c6881dd01666ec331e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.048157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.049836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 155/308] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/users/155802b7e89c44c6881dd01666ec331e => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:19.051552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bffba-0475-470c-b409-74d952134909 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.055996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.139280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.173920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58a04-5f8b-4004-953a-1866327ee26e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.183441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bffba-0475-470c-b409-74d952134909 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.186005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.217524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c578b-7fb9-4605-806e-b04e934bf817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.218303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c578b-7fb9-4605-806e-b04e934bf817 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.218586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c578b-7fb9-4605-806e-b04e934bf817 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.218881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c578b-7fb9-4605-806e-b04e934bf817 None admin] PATH_INFO: `/v3/system/users/155802b7e89c44c6881dd01666ec331e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.220374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c578b-7fb9-4605-806e-b04e934bf817 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=155802b7e89c44c6881dd01666ec331e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.221653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c578b-7fb9-4605-806e-b04e934bf817 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.227595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.227595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.227944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.228253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.229026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] RBAC: Adding query filter params (role.id=b7d97d6c79bf4e0c90bce6aec6777332, user.id=2e70eae957dd4b36ad4d6eba22465d16, scope.system=all) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:19.229314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.230723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.231909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 156/309] 10.222.0.72 () {58 vars in 1334 bytes} [Mon Sep 26 17:45:19 2022] PUT /identity/v3/system/users/155802b7e89c44c6881dd01666ec331e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:19.232847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bf735-2d42-4b35-a378-dddc92f209a9 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.233201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bf735-2d42-4b35-a378-dddc92f209a9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.233484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bf735-2d42-4b35-a378-dddc92f209a9 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.239200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 157/310] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:19.241233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.241514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.241788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.248018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 154/311] 10.222.0.72 () {56 vars in 1308 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/role_assignments?user.id=2e70eae957dd4b36ad4d6eba22465d16&scope.system=all&role.id=b7d97d6c79bf4e0c90bce6aec6777332 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:19.265775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:19.267407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:19.353009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 158/312] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:19.629007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.740178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c066fa2-86ff-4fa3-be1b-5fdbb645943c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.777509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.778532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.779033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.779497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.781418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.792053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:19.797154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:19.798026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:19.798874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:19.806740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 155/313] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:19.808339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.913285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef09c0-ebf1-4da1-b2a6-50cdddde0d15 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.915971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.951372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.951981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.952257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.952528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.953190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:19.953540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.954942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.968654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 159/314] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:19.970131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.077553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eecdf-ea59-4542-9031-14636c6ea37a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:20.115220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:20.115862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.116140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.116406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] PATH_INFO: `/v3/users/74503ee96439434faad6a961ce6ddf91` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.117551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] RBAC: Authorizing `identity:get_user(user_id=74503ee96439434faad6a961ce6ddf91)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:20.119032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:20.120612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 156/315] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/users/74503ee96439434faad6a961ce6ddf91 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:20.126331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.235834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db045a-d83b-4fd8-bac7-644836356342 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:20.270320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:20.271490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.271765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.272030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.305741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:20.340479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:20.341929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:20.440203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 160/316] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:20.446806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.513908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:20.554783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2668-88b8-4925-90c4-c5ea57b9824b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:20.591848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056805d9-5b7e-4b23-abfe-dab092f1bd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:20.592483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056805d9-5b7e-4b23-abfe-dab092f1bd7b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.592758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056805d9-5b7e-4b23-abfe-dab092f1bd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.593028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056805d9-5b7e-4b23-abfe-dab092f1bd7b None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.594135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056805d9-5b7e-4b23-abfe-dab092f1bd7b None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:20.595598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056805d9-5b7e-4b23-abfe-dab092f1bd7b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:20.597226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 157/317] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:20 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:20.603934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.675877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:20.714541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4477e-6525-4ffe-b475-994e797f8adf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:20.750090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:20.750734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.751043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.751367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.752086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] RBAC: Adding query filter params (role.id=b7d97d6c79bf4e0c90bce6aec6777332, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446, user.id=74503ee96439434faad6a961ce6ddf91) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:20.752441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:20.753926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59e7c5-3055-42e6-bb87-555c94400fe8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:20.760742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 161/318] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:20 2022] GET /identity/v3/role_assignments?user.id=74503ee96439434faad6a961ce6ddf91&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=b7d97d6c79bf4e0c90bce6aec6777332 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:21.173025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34149e-13ce-4fda-992c-f86a9e29ffe3 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.173561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34149e-13ce-4fda-992c-f86a9e29ffe3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.173561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34149e-13ce-4fda-992c-f86a9e29ffe3 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.174968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 158/319] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:21.181206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4aa11-bdcf-427c-9834-c179c7a55c8c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.181660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4aa11-bdcf-427c-9834-c179c7a55c8c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.181732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4aa11-bdcf-427c-9834-c179c7a55c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.200143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-91b4aa11-bdcf-427c-9834-c179c7a55c8c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:21.201319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-91b4aa11-bdcf-427c-9834-c179c7a55c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:21.270018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 162/320] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:21.453617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a93e41-742d-471d-af52-2e701d5684a3 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.454014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a93e41-742d-471d-af52-2e701d5684a3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.454231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a93e41-742d-471d-af52-2e701d5684a3 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.455778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 159/321] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:21.461457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a898a3-0115-41d0-aafd-6b70b58d3441 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.461740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a898a3-0115-41d0-aafd-6b70b58d3441 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.462018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a898a3-0115-41d0-aafd-6b70b58d3441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.480575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-85a898a3-0115-41d0-aafd-6b70b58d3441 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:21.481752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-85a898a3-0115-41d0-aafd-6b70b58d3441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:21.540850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:21.545296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a93e41-742d-471d-af52-2e701d5684a3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:21.554316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 163/322] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:21.559304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250419-e45f-436d-b494-6306899e1ffe None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.559599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250419-e45f-436d-b494-6306899e1ffe None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.559874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250419-e45f-436d-b494-6306899e1ffe None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.561319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 164/323] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:21.566270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.566557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.566829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.586006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:21.587236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:21.651271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a93e41-742d-471d-af52-2e701d5684a3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:21.659254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 165/324] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:21.683846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:21.692342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:21.693544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.693928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.694481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.695567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:21.696986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:21.698697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 160/325] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:21.704164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:21.807007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc495af0-f8c9-472c-8237-80ad6538e28a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:21.828870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:21.829728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:21.844665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:21.845282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.845562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.845837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] PATH_INFO: `/v3/groups` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.846483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:21.847961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:21.865401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 166/326] 10.222.0.72 () {60 vars in 1134 bytes} [Mon Sep 26 17:45:21 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:21.935261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:21.941839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b4de-81de-46c6-9fd3-60f4f33a22d1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:21.991665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:21.993169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.993758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.994220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.009939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:22.014969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:22.015922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:22.016750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 161/327] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:22.022390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.058022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031f079-ef46-45e9-b3bb-7c4cfd846d2b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.092869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:22.098461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.099894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.100184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.100451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.111484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:22.116968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:22.118060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:22.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 167/328] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:22.123806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.151748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b464eb-1527-4285-9a9e-e9dc56af463f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.180386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:22.201585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.202252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.202591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.202923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.203542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:22.203852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.205350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.216106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 162/329] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:22.221655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.267109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e07a5-ff47-4709-8198-f20ab022e3cc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.315091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.315630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.315914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.316191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.316869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:22.317245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.319793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.331855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 168/330] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:22.334557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.371414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002b530-117f-4eda-a69b-b947e1300e53 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.396190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:22.414971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.415397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.415675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.415948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] PATH_INFO: `/v3/users/155802b7e89c44c6881dd01666ec331e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.417080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] RBAC: Authorizing `identity:get_user(user_id=155802b7e89c44c6881dd01666ec331e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.418525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.420449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 163/331] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/users/155802b7e89c44c6881dd01666ec331e => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:22.426581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.483345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff60b67-ad5e-4b3c-9e62-82cbdf4c75f0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.485931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:22.535677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.537752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.538265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.539083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] PATH_INFO: `/v3/users/2e70eae957dd4b36ad4d6eba22465d16` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.541058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] RBAC: Authorizing `identity:get_user(user_id=2e70eae957dd4b36ad4d6eba22465d16)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.542489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.544493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 169/332] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/users/2e70eae957dd4b36ad4d6eba22465d16 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:22.551002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.570513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ed81d-e9fc-47f2-be76-31ef3c5301aa None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.612546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.613923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.615002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.615504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.616780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] RBAC: Adding query filter params (role.id=7642cb8c018c4602b0a15c0e67781890, user.id=155802b7e89c44c6881dd01666ec331e, scope.system=all) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:22.617311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.619175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92e509-1d7a-4be6-b32d-324cb5f84ecb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.635942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 164/333] 10.222.0.72 () {56 vars in 1308 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/role_assignments?user.id=155802b7e89c44c6881dd01666ec331e&scope.system=all&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:22.679772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794c514-d667-4794-9a43-ebdf3f5029fc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.722430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1f55-fec5-40a2-94f8-bd8b061fca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.723131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1f55-fec5-40a2-94f8-bd8b061fca11 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.723408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1f55-fec5-40a2-94f8-bd8b061fca11 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.723714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1f55-fec5-40a2-94f8-bd8b061fca11 None admin] PATH_INFO: `/v3/system/users/2e70eae957dd4b36ad4d6eba22465d16/roles/b7d97d6c79bf4e0c90bce6aec6777332` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.725858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1f55-fec5-40a2-94f8-bd8b061fca11 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2e70eae957dd4b36ad4d6eba22465d16, role_id=b7d97d6c79bf4e0c90bce6aec6777332)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.727289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1f55-fec5-40a2-94f8-bd8b061fca11 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.740835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 170/334] 10.222.0.72 () {58 vars in 1334 bytes} [Mon Sep 26 17:45:22 2022] PUT /identity/v3/system/users/2e70eae957dd4b36ad4d6eba22465d16/roles/b7d97d6c79bf4e0c90bce6aec6777332 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:24.339867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8447e-9c44-4b5e-88b8-038bc4d8d1de None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.339867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8447e-9c44-4b5e-88b8-038bc4d8d1de None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.339867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8447e-9c44-4b5e-88b8-038bc4d8d1de None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.339867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 165/335] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:24.345541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f6fe-9e82-4646-b216-4f5f4d59888f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.346377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f6fe-9e82-4646-b216-4f5f4d59888f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.346884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f6fe-9e82-4646-b216-4f5f4d59888f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.370967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3807f6fe-9e82-4646-b216-4f5f4d59888f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:24.372510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3807f6fe-9e82-4646-b216-4f5f4d59888f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:24.408910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.467766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 171/336] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:24.635023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fa848-fd54-41d9-99d3-6096d25a63ee None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.635319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fa848-fd54-41d9-99d3-6096d25a63ee None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.635540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fa848-fd54-41d9-99d3-6096d25a63ee None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.637182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 166/337] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:24.642213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.642532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.642814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.662349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:24.663568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:24.749116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 172/338] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:24.904653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fa848-fd54-41d9-99d3-6096d25a63ee None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.969210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.047591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fa848-fd54-41d9-99d3-6096d25a63ee None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.092819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.093478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.093810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.094025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.109491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:25.115651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:25.116510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:25.117283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:25.118352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 167/339] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:25.123050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.253951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.259457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.277865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962c36-5e80-4b7a-9e43-b12c71673f5e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.352528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.353875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.354324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.354823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.355749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:25.356272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.357872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.374461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 173/340] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:25.383412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.482002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220c0c7-a940-46c6-abbc-bf03d047cdcf None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.494578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.571168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.571736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.572013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.572282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.596139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:25.606064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:25.607062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:25.607902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 168/341] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:25.619544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.634436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bc6d1-6dc6-405f-bd1b-0821d7e92020 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.674610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8d766-5923-4704-a55f-c988c6d14e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.675492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8d766-5923-4704-a55f-c988c6d14e4d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.675949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8d766-5923-4704-a55f-c988c6d14e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.676376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8d766-5923-4704-a55f-c988c6d14e4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.708909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.779041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8d766-5923-4704-a55f-c988c6d14e4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.780812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8d766-5923-4704-a55f-c988c6d14e4d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.825712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7602864-f7d5-4012-b124-df4367b175a7 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.887905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.888534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.888818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.889093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.889762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:25.890069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.891489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d1267-273a-4915-beb8-dd3d719c87c7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.903983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 169/342] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:25.905536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167e02-7e62-41ae-8522-62575e2595e6 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.905978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167e02-7e62-41ae-8522-62575e2595e6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.906478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167e02-7e62-41ae-8522-62575e2595e6 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.908760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 170/343] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:25.910038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332973f-d7f3-4f75-8557-84e1866b7d8e None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.910536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332973f-d7f3-4f75-8557-84e1866b7d8e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.911142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332973f-d7f3-4f75-8557-84e1866b7d8e None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.913476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 171/344] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:25.914411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332973f-d7f3-4f75-8557-84e1866b7d8e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.925529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 174/345] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:25.927691 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294736-449a-4967-9d21-250f3d8f03c0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.928279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294736-449a-4967-9d21-250f3d8f03c0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.928754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294736-449a-4967-9d21-250f3d8f03c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.948811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ef294736-449a-4967-9d21-250f3d8f03c0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:25.950359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ef294736-449a-4967-9d21-250f3d8f03c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:25.977752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:26.015098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:26.038345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 175/346] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:26.040747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.041032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.041307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.078314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:26.079482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:26.106460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332973f-d7f3-4f75-8557-84e1866b7d8e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:26.166744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:26.167556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.167848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.168128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.210663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 176/347] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:26.211612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.261582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:26.263122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:26.300081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:26.390998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17d54e-904e-4ab7-8717-644714d732b1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:26.422266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:26.436827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 172/348] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:26.439512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.465815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:26.466465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.466743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.467037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.470942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:26.473438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:26.480162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 177/349] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:26.481040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.617626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074f4a1-5a3c-4eb2-b9e1-738fa10f123a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:26.659576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5517af-fbf6-4172-bfb6-e9f8d2485a9d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:26.674171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:26.687568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:26.689926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.690418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.691073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.694027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:26.696388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:26.700621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 173/350] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:26.701268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.734365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:26.735421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.735919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.736400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.751337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:26.756836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:26.757790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:26.758605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:26.761161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 178/351] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:26.762222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.831779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:26.839342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05df440-c63a-4ee5-9845-ab59822e5635 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:26.880728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:26.881722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.882325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.882797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.890691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:26.892496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:26.894714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 174/352] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:26.896331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.903454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac55313-5e9f-4a6b-a7dc-c16cd8bcfeca None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:26.941880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:26.943212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:26.943563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:26.943830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:26.945343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:26.946952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:26.948858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 179/353] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:26.949741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:26.964840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.039522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103342b-deb5-4e35-b932-74de88bac63f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.087732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.088484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.088832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.089185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.107203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:27.112844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:27.113744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:27.115394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 175/354] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:27.116106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.128749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251815b-1dcc-4d7a-ba9b-07db5853c8d3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.150619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.173261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.174261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.174549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.174995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.175665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:27.175971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.177993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.187725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 180/355] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:27.188415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.240711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28344234-894a-4d5e-9825-79b832a41d7e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.266244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.281804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.282570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.282872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.283264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.283998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:27.284310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.285766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.292266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 176/356] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:27.293470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.305286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20f0c6-bd31-4fe7-9b54-5c7b71c6f42f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.355513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.356204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.356487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.356760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.357501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:27.357809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.359478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef22247-6baf-4810-8fce-5430a05244f9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.367312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 181/357] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=238ba40fcefe4923bb27057b1074fbef => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:27.370886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a367d1-73e6-417d-b1e7-1113fc7c94ce None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.371051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a367d1-73e6-417d-b1e7-1113fc7c94ce None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.371271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a367d1-73e6-417d-b1e7-1113fc7c94ce None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.373064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 182/358] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:27.374698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2e3bc-cea3-44da-ab8e-329b64a51e1b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.375094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2e3bc-cea3-44da-ab8e-329b64a51e1b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.375369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2e3bc-cea3-44da-ab8e-329b64a51e1b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.380636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 183/359] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:27.381902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2e3bc-cea3-44da-ab8e-329b64a51e1b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.443353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.452715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783a839-7e55-4390-820b-6c83a20cc444 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.503853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.506025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.506654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.507126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.507984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:27.508459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.510827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a867d-1d36-417b-a337-bdf476c7dd9d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.524233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 177/360] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:27.525882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c512d5d-12be-4f99-a03c-9f63b78716ca None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.526010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2e3bc-cea3-44da-ab8e-329b64a51e1b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.526209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c512d5d-12be-4f99-a03c-9f63b78716ca None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.526438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c512d5d-12be-4f99-a03c-9f63b78716ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.550181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4c512d5d-12be-4f99-a03c-9f63b78716ca None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:27.551840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4c512d5d-12be-4f99-a03c-9f63b78716ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:27.576320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.577076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.577362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.577639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.605031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.635502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 178/361] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:27.636924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.637218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.637494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.655778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:27.657072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:27.673059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.674795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.726740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.741003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 179/362] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:27.741918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.851967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 184/363] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:27.858699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.930133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e704a8-7c46-4aed-a3ef-d48ca2d3c057 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.939705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.001305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.002010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.002291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.002566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] PATH_INFO: `/v3/users/2e70eae957dd4b36ad4d6eba22465d16` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.003820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] RBAC: Authorizing `identity:get_user(user_id=2e70eae957dd4b36ad4d6eba22465d16)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.005634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.009313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 180/364] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/users/2e70eae957dd4b36ad4d6eba22465d16 => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.015202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.066002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12338c5-6081-412e-b29c-7af7a74cda0d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.136117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.136687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.137037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.137318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.139536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.141009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.144744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 185/365] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.150850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.162555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.198552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6920-321a-42a9-8c23-589ab6287a63 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.245173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.245889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.246180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.246457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.247342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] RBAC: Adding query filter params (role.id=b7d97d6c79bf4e0c90bce6aec6777332, user.id=2e70eae957dd4b36ad4d6eba22465d16, scope.system=all) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.247769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.249555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.268747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 181/366] 10.222.0.72 () {56 vars in 1308 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/role_assignments?user.id=2e70eae957dd4b36ad4d6eba22465d16&scope.system=all&role.id=b7d97d6c79bf4e0c90bce6aec6777332 => generated 510 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.269629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.324697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bb5d9-7d7c-4d2d-a442-761678689d90 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.360875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.410666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.411924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.412221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.412555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.417394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.419554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.426860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 186/367] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.426860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.544505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.553792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f77d0-3a7a-490f-990e-206b012a0454 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.587117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfeda6-042e-42ac-8b5c-cebf6eebd4fb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.616895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.617885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.618175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.618445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.640760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:28.646304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.646989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.647318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.647670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:28.648447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:28.649472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:28.650366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 182/368] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:28.651589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.668291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:28.673933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:28.676127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 187/369] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:28.683129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.787546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.801303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf0988-5a96-4f79-8057-6f00a4ede154 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.826992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48c95a-58a0-423a-bc78-f39284993d53 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.848057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.849268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.850365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.850365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.851332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.851643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.853313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.862534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 183/370] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.866265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.885728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.886716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.887160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.887606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.888480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.888986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.890530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.892583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.908040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 188/371] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.915600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.001920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15667a-afe6-4dc5-905e-5479fafb0cf4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.050124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.050774 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.051162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.051444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.053753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:29.054160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.055795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.065109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 184/372] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:29.076366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.125952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.136400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09b8ce-3f8b-46d9-b96d-e462eac1845d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.221181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.221830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.222112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.222383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.256209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202815-5275-496f-ac19-2ac0d0ff573c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.294683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.303605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.305285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.343767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.344403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.344686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.344960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.419916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 189/373] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:29.429422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.495273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.502412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.536882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.622547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.636988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78174c-195e-4454-926d-9bbd2f053387 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.695661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 185/374] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:29.702015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.704220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.705781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.706327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.706920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.713239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.714991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.716918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 190/375] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:29.722294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.882043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aeaa06-218d-4d3c-8ff9-c7c69409539e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.902258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.910674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53920730-6bb7-481a-84a3-246b3cf8ace1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.925380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.926027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.927123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.927123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] PATH_INFO: `/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.928245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] RBAC: Authorizing `identity:get_project(project_id=bbe873ec2e4b4dc7b12dd87b247ac3dc)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.929692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.934860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 186/376] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:29.936690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.988194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.988838 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.989167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.989998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.991573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.993446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.995556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 191/377] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:30.002456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.066796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.154965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608748ed-db6a-497c-ae6b-cca35582846a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.168738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc281c8c-f09c-410f-9067-2afbe1b16492 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.220595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.222440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.223240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.224636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.225887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:30.226685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f7ca8-879e-4e39-a770-69e9da89ba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.227564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7ca8-879e-4e39-a770-69e9da89ba0f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.228103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.229737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.230223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7ca8-879e-4e39-a770-69e9da89ba0f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.230697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7ca8-879e-4e39-a770-69e9da89ba0f None admin] PATH_INFO: `/v3/groups/e9577aa276904d5aa438f8ac83528388` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.240333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f7ca8-879e-4e39-a770-69e9da89ba0f None admin] RBAC: Authorizing `identity:get_group(group_id=e9577aa276904d5aa438f8ac83528388)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.243523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 192/378] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:30 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=238ba40fcefe4923bb27057b1074fbef => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:30.244462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f7ca8-879e-4e39-a770-69e9da89ba0f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.246785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 187/379] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/groups/e9577aa276904d5aa438f8ac83528388 => generated 229 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:30.252719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.267020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec3de7-08f5-49d3-b093-075537f5706d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.267696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec3de7-08f5-49d3-b093-075537f5706d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.268183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec3de7-08f5-49d3-b093-075537f5706d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.270584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 188/380] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:30.278576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8bb8e-b7d8-49bb-9882-c6ed94377d03 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.278949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8bb8e-b7d8-49bb-9882-c6ed94377d03 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.279179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8bb8e-b7d8-49bb-9882-c6ed94377d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.314742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-73f8bb8e-b7d8-49bb-9882-c6ed94377d03 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:30.316019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-73f8bb8e-b7d8-49bb-9882-c6ed94377d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:30.431836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.451936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.462379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 189/381] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:30.463498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb454b-9fff-4f1e-813d-20fd1d66c7be None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.463781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb454b-9fff-4f1e-813d-20fd1d66c7be None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.465272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb454b-9fff-4f1e-813d-20fd1d66c7be None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.466252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34982d9-19c8-4d08-8096-ecab5ab09c48 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.469372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 190/382] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:30.476911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.477202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.477470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.508149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:30.509388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:30.527461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.528491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.529160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.530076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.531390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] RBAC: Adding query filter params (group.id=e9577aa276904d5aa438f8ac83528388, role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=bbe873ec2e4b4dc7b12dd87b247ac3dc) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:30.531910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.534256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.542918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 193/383] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:30 2022] GET /identity/v3/role_assignments?group.id=e9577aa276904d5aa438f8ac83528388&scope.project.id=bbe873ec2e4b4dc7b12dd87b247ac3dc&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:30.590962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 191/384] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:30.797950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.924480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a4056-778b-42a0-95c0-5c3d0400bc1f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.971594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.972249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.972531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.972803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.988528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:30.992502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:30.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:30.995496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:31.000916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 194/385] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:31.001612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.032982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.124121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67615c-6a3a-4d5d-b1a7-ab7e9c826733 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.151011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.166777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8cc92-8ecd-40ea-ae47-0e1e252a9272 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.202865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.203514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.203856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.204136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.223052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:31.229497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:31.230373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:31.231241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:31.231871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.231871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.231871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.231871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.232675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:31.232777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.234083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.243782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 195/386] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:31.244058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 192/387] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:31.249629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.249777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.406312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d5469-de17-4dbf-a34d-c0ceef95a650 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.454556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.455236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.455488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.455767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.456435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:31.456740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.458173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.460762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.483268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 193/388] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:31.488880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.492401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54514b4a-3a1d-448d-8ae7-bc36afc50890 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.551119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.568039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.569069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.569815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.570302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.573939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.575805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.577607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 196/389] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:31.582431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.650826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33a6d5-1b6c-4282-99c2-63dc55b6eb32 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.698061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.699702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.699985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.700279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.702359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.703843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.711123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 194/390] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:31.716212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.806658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ca63f-6394-4aab-8447-643d3704debe None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.850103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.854060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4d755-c912-4cf7-b8f7-5436eb96e443 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.879554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.880296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.880594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.880845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.882043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.883550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.885494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 197/391] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:31.894860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.903862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.904900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.905395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.905894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.908339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.909843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.914666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 195/392] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:31.924181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.928678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:32.100139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1fb40-091e-4f00-b443-ae6e224a6e2d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.108853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40b764-482b-4674-824c-207e597ed2f1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.148026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02f302-ccf5-4a44-9189-c1010385cd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.148697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f302-ccf5-4a44-9189-c1010385cd84 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.148981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f302-ccf5-4a44-9189-c1010385cd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.149259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f302-ccf5-4a44-9189-c1010385cd84 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/groups/3970acbe616c460d952bd33f73f19817/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02f302-ccf5-4a44-9189-c1010385cd84 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee6e468e90c34c0680f4b1a15b0f5055, group_id=3970acbe616c460d952bd33f73f19817, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.152907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02f302-ccf5-4a44-9189-c1010385cd84 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.168465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100a69a-d570-4730-9a01-baaabedc273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.169489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100a69a-d570-4730-9a01-baaabedc273e None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.169987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100a69a-d570-4730-9a01-baaabedc273e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.170240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 196/393] 10.222.0.72 () {58 vars in 1442 bytes} [Mon Sep 26 17:45:31 2022] PUT /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/groups/3970acbe616c460d952bd33f73f19817/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:45:32.170932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100a69a-d570-4730-9a01-baaabedc273e None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/groups/3970acbe616c460d952bd33f73f19817/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.180484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100a69a-d570-4730-9a01-baaabedc273e None admin] RBAC: Authorizing `identity:create_grant(project_id=ee6e468e90c34c0680f4b1a15b0f5055, group_id=3970acbe616c460d952bd33f73f19817, role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.181972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100a69a-d570-4730-9a01-baaabedc273e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.209927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 198/394] 10.222.0.72 () {58 vars in 1442 bytes} [Mon Sep 26 17:45:31 2022] PUT /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055/groups/3970acbe616c460d952bd33f73f19817/roles/238ba40fcefe4923bb27057b1074fbef => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:32.213961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47024-7975-498c-b3b1-005ce088c3e5 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.214320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47024-7975-498c-b3b1-005ce088c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.214541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47024-7975-498c-b3b1-005ce088c3e5 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.216478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 197/395] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:32.222935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.223249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.223840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.264301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:32.265719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:32.358234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 199/396] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:32.651989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47024-7975-498c-b3b1-005ce088c3e5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.685297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:32.784452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47024-7975-498c-b3b1-005ce088c3e5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.825725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02747ded-e0ee-46bd-b745-48bc12bb48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.826359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02747ded-e0ee-46bd-b745-48bc12bb48f3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.826635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02747ded-e0ee-46bd-b745-48bc12bb48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.826930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02747ded-e0ee-46bd-b745-48bc12bb48f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.847748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02747ded-e0ee-46bd-b745-48bc12bb48f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-02747ded-e0ee-46bd-b745-48bc12bb48f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:32.853238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:32.854132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:32.855606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 198/397] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:32.860202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.877000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.005755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce29f3b-951a-492c-a962-0d86911d9273 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.061415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.062047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.062325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.062591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.063283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:33.063592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:33.064966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:33.067391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b6e45-d052-44e1-8f0f-81b1509c4332 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.067671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b6e45-d052-44e1-8f0f-81b1509c4332 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.067941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b6e45-d052-44e1-8f0f-81b1509c4332 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.069786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 199/398] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:33.074297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.074993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.075273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.081982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 200/399] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:33.087847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.097320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:33.099240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:33.196284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 200/400] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:33.219519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.262303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fd498-5847-4ffb-ab45-c661372bd271 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.297900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.298583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.298892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.299172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.300284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:33.302459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:33.304573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 201/401] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:33.310054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.334974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.441591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ae6aa-b410-452e-bb69-0a173d7eebdb None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.493985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.494679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.494991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.495265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.496403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:33.497845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:33.499529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 201/402] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:33.506093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.556324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.631847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed344c7e-61e1-4bfc-9000-d9f7ac1d10dc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.660314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.670170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3329cf-0623-42cd-9e0a-3ce7b68b6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.670782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3329cf-0623-42cd-9e0a-3ce7b68b6cd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.671111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3329cf-0623-42cd-9e0a-3ce7b68b6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.671388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3329cf-0623-42cd-9e0a-3ce7b68b6cd4 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/groups/3970acbe616c460d952bd33f73f19817/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.673332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3329cf-0623-42cd-9e0a-3ce7b68b6cd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8ffdfac9ab4b53a043e4aa806e5446, group_id=3970acbe616c460d952bd33f73f19817, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:33.674716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3329cf-0623-42cd-9e0a-3ce7b68b6cd4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:33.680214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b68a12-032c-4247-be34-7bb2880ac712 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.685501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.692978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 202/403] 10.222.0.72 () {58 vars in 1442 bytes} [Mon Sep 26 17:45:33 2022] PUT /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/groups/3970acbe616c460d952bd33f73f19817/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:33.696431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de3242-1427-4702-9ef8-41305439a531 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.696851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de3242-1427-4702-9ef8-41305439a531 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.697204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de3242-1427-4702-9ef8-41305439a531 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.698875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 203/404] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:33.708166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.709058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.709447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.728677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.729385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.729673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.730000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.734400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:33.735734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:33.744912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:33.750056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:33.751027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:33.751811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 202/405] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:33.759014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.851063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 204/406] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:33.943934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5c782-0549-4281-ac20-af0904c25543 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.999193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.999815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.000097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.000372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.001039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:34.001344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.002730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.014138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 203/407] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.019762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.166669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45289a-16ad-4d6b-b8b8-f59c0e0bfb5a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.197144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.209962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.210661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.211103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.211438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.212736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.214173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.216065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 205/408] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.221544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.237731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.256418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.372531 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf140a7f-d29f-4771-8ee8-a29973f3ec34 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.440148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.440705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.440983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.441256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.443680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.445393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.447127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 204/409] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.452763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb440d8-6f4b-4035-bad1-ba0026b47f14 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.456937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.502361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.503020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.503318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.503573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.515310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:34.521334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:34.522268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:34.523122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 206/410] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:34.528381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.591284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.595930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.639486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742d0c-3527-4c4e-8089-15d6a6ee682f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.639908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814fbd4-7455-453f-b022-66a9c53e0f5e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.675874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.677214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.677489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.677926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.678651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:34.678983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.680542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.689612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 207/411] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.695990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.705519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.706210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.706493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.706775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/groups/3970acbe616c460d952bd33f73f19817/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.712818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8ffdfac9ab4b53a043e4aa806e5446, group_id=3970acbe616c460d952bd33f73f19817, role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.714272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.741937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 205/412] 10.222.0.72 () {58 vars in 1442 bytes} [Mon Sep 26 17:45:34 2022] PUT /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446/groups/3970acbe616c460d952bd33f73f19817/roles/238ba40fcefe4923bb27057b1074fbef => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:34.811195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a3328-bd52-4dec-b723-96db28243356 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.834210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.849410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148e108-8e3c-4673-85ef-cdc8f1871a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.850055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e108-8e3c-4673-85ef-cdc8f1871a34 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.850398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e108-8e3c-4673-85ef-cdc8f1871a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.850678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148e108-8e3c-4673-85ef-cdc8f1871a34 None admin] PATH_INFO: `/v3/groups/e9577aa276904d5aa438f8ac83528388` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.852169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e108-8e3c-4673-85ef-cdc8f1871a34 None admin] RBAC: Authorizing `identity:get_group(group_id=e9577aa276904d5aa438f8ac83528388)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.853651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148e108-8e3c-4673-85ef-cdc8f1871a34 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.855580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 208/413] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/groups/e9577aa276904d5aa438f8ac83528388 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.861342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.011747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c10d3-72cd-4764-9129-8cabe15635ce None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.028108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d75bd8-41e1-4e71-99a4-15d2a007b352 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.028635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d75bd8-41e1-4e71-99a4-15d2a007b352 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.028635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d75bd8-41e1-4e71-99a4-15d2a007b352 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.029991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 209/414] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:35.035709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4ab1f-6177-4991-ad2e-21300c1eb0ef None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.036610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4ab1f-6177-4991-ad2e-21300c1eb0ef None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.036886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4ab1f-6177-4991-ad2e-21300c1eb0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.064133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.065553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.066406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.067216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] PATH_INFO: `/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.070328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe873ec2e4b4dc7b12dd87b247ac3dc)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.070530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6ff4ab1f-6177-4991-ad2e-21300c1eb0ef None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:35.071755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6ff4ab1f-6177-4991-ad2e-21300c1eb0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:35.073950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.076732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 206/415] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:35.091191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.097130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.190657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 210/416] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:35.198125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7489731-3210-4cad-9fbe-8f50915d8de9 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.198419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7489731-3210-4cad-9fbe-8f50915d8de9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.198698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7489731-3210-4cad-9fbe-8f50915d8de9 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.200293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 211/417] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:35.216817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8a478-3010-4cf9-9cb5-cfce1bdce556 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.217432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.217724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.217999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.251304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:35.252423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:35.256558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.257231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.257801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.258080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] PATH_INFO: `/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc/groups/e9577aa276904d5aa438f8ac83528388/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.261010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe873ec2e4b4dc7b12dd87b247ac3dc, group_id=e9577aa276904d5aa438f8ac83528388, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.262482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.282522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 207/418] 10.222.0.72 () {58 vars in 1442 bytes} [Mon Sep 26 17:45:35 2022] PUT /identity/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc/groups/e9577aa276904d5aa438f8ac83528388/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:35.373687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 212/419] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:35.602094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.671494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.718478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6fb1-e69d-454f-b0b3-998bdd5e39dd None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.744860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.756197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.756899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.757185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.757456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.769323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:35.775426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:35.777521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:35.779483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:35.787241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 208/420] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:35.788190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.827357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.927144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6442aa-1141-49e9-a848-ce7cbd8eec26 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.972057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108f47-8f19-455e-aea1-1e795d978aa2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.003633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.007559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.008214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.008494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.008767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.015500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.016110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.016401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.016676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.017343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:36.017650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.019061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.028405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 209/421] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:36.033821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.037005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:36.051222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:36.052989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:36.053804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:36.056853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 213/422] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:36.061416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.229081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c937d9-c6ec-4a68-817e-1682b9109a21 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.268154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d078fe7-e214-4b5f-bfd3-f4c40accaa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.268931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078fe7-e214-4b5f-bfd3-f4c40accaa2a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.269182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078fe7-e214-4b5f-bfd3-f4c40accaa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.269467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d078fe7-e214-4b5f-bfd3-f4c40accaa2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.272979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9037b8d-2abb-45ef-9676-93cb46b8663b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.288696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.340729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.355576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.356205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.356488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.356761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.357440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] RBAC: Adding query filter params (name=member) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:36.357744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.357930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d078fe7-e214-4b5f-bfd3-f4c40accaa2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.359275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.359452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d078fe7-e214-4b5f-bfd3-f4c40accaa2a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.377408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 214/423] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:36.383351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.476883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 210/424] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:45:36.483073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caffa3-363e-4a8e-8d5b-2d4d8a76de87 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.483073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caffa3-363e-4a8e-8d5b-2d4d8a76de87 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.483073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caffa3-363e-4a8e-8d5b-2d4d8a76de87 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.483073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 211/425] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:36.484769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caffa3-363e-4a8e-8d5b-2d4d8a76de87 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.544476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fe082-f558-4c0c-8704-fd1faa9c7756 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.582473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.594995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.596333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.597140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.597655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.607094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caffa3-363e-4a8e-8d5b-2d4d8a76de87 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.648967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.651159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395287a-fd5d-409b-bf8f-6ab12d138774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.651808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395287a-fd5d-409b-bf8f-6ab12d138774 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.652092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395287a-fd5d-409b-bf8f-6ab12d138774 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.652365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395287a-fd5d-409b-bf8f-6ab12d138774 None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.653527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395287a-fd5d-409b-bf8f-6ab12d138774 None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.654993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395287a-fd5d-409b-bf8f-6ab12d138774 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.662394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 212/426] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:36.662394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.662394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.662394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.682300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:36.683723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:36.700999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.702464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.779243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 213/427] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:36.779934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.839892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 215/428] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:45:36.845166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.946805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.947158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.982025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddae758-4385-4e6c-96cf-aab9191aa294 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.017811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c0c5d-8c20-477b-9789-778dd7f8bf3c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.050408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.052685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.052968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.053246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.057503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.058976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:37.062866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 214/429] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:37.068908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.092812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.093531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.093854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.094132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] PATH_INFO: `/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.095454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] RBAC: Authorizing `identity:get_project(project_id=ee6e468e90c34c0680f4b1a15b0f5055)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.102871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:37.102871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 216/430] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/projects/ee6e468e90c34c0680f4b1a15b0f5055 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:37.106014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.216952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf590b1-20a5-47ae-a84c-878f19bd7aa0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.228890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d639c-fb1c-4304-ab4a-d50e3ba6553d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.251957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.252555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.252835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.253108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.253826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:37.254135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.255685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:37.261616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 215/431] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=238ba40fcefe4923bb27057b1074fbef => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:37.263594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.266192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecdcfd-fed7-4e96-b468-db981f31e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.266923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecdcfd-fed7-4e96-b468-db981f31e902 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.267211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecdcfd-fed7-4e96-b468-db981f31e902 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.267490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecdcfd-fed7-4e96-b468-db981f31e902 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.268682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecdcfd-fed7-4e96-b468-db981f31e902 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.270771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecdcfd-fed7-4e96-b468-db981f31e902 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:37.278787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 217/432] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:37.280931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56480a3d-9e92-46b7-a40e-895279ff7772 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.280931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56480a3d-9e92-46b7-a40e-895279ff7772 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.281123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56480a3d-9e92-46b7-a40e-895279ff7772 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.287139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 218/433] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:37.287139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56480a3d-9e92-46b7-a40e-895279ff7772 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.289915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:37.303504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:37.412481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56480a3d-9e92-46b7-a40e-895279ff7772 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.413029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c99367-0386-4bd9-9ffd-7d3cb0346b9f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.453932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.454556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.454869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.454967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.455144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.455501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.455860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.456131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.456942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:37.457245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.459073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632df954-7f63-4e75-96d8-36de7c10ad3b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:37.465612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 219/434] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=ee6e468e90c34c0680f4b1a15b0f5055&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:37.467704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:37.472293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.472577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.472847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:37.473791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:37.474682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:37.475666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 216/435] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:37.480877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.502524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:37.503949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:37.585709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:37.593002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 220/436] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:37.597020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:37.628632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc2cb6-e4ea-44e6-9d35-815288ebe2c2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.671630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.672257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.672542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.672821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.673519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] RBAC: Adding query filter params (name=member) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:37.673828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.675280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:37.691569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 217/437] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:37.698460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.812671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38482bf-0beb-47a3-a42e-317a81a55c49 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:37.854887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820408f8-e68f-4c7a-8bfb-b365190da367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:37.855975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820408f8-e68f-4c7a-8bfb-b365190da367 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.856388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820408f8-e68f-4c7a-8bfb-b365190da367 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.856813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820408f8-e68f-4c7a-8bfb-b365190da367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.884563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:37.935371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820408f8-e68f-4c7a-8bfb-b365190da367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:37.938107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:37.938496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820408f8-e68f-4c7a-8bfb-b365190da367 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.015034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487336af-f0ce-4b1b-ac66-5e49de86186d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.055849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cfa81-83a8-4448-a28c-ccc5431087ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.056684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cfa81-83a8-4448-a28c-ccc5431087ec None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.056803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 221/438] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:38.056890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cfa81-83a8-4448-a28c-ccc5431087ec None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.057030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cfa81-83a8-4448-a28c-ccc5431087ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.057736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601db196-f4e3-4a6f-9a5c-13c7794731b6 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.058026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601db196-f4e3-4a6f-9a5c-13c7794731b6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.058455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601db196-f4e3-4a6f-9a5c-13c7794731b6 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.059934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 222/439] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:38.065634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601db196-f4e3-4a6f-9a5c-13c7794731b6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.066588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.071794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cfa81-83a8-4448-a28c-ccc5431087ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-ee2cfa81-83a8-4448-a28c-ccc5431087ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:38.077201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:38.078108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:38.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 218/440] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:38.079656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.085323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.085323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.099357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:38.100821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:38.182820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 219/441] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:38.183642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.190207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.196712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601db196-f4e3-4a6f-9a5c-13c7794731b6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.230690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.231528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.231804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.232090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.233160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.234590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.236246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 223/442] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:38.242283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.293947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e0821-0396-4e6d-9359-0d7cebcbf805 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.322428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.332343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.333645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.333922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.334191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.334856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.335172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.336527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.349690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 220/443] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:38.349841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649e4e3-8229-4201-82eb-a9f63cbf5c4a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.351752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.384732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.385360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.385642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.385912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.387041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.388463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.390035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 224/444] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.396282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.429891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.459588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8b4db-57b6-4248-a62b-d71c54bd77c7 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.495289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.495897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.496184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.496457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.498649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d854-3259-4d87-862b-12a0c44d32c7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.532880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.533482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.533791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.534063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.534781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=7642cb8c018c4602b0a15c0e67781890, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.535120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.536701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.542406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 225/445] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=7642cb8c018c4602b0a15c0e67781890 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.543212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.565440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.580105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.581593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.664422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.668965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa603dd-b139-4e67-9bce-91856238808a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.706585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 221/446] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:45:38.708199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.708581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.709334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.710003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.721306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:38.728086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:38.729005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:38.729818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 226/447] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:38.736525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.821616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.835990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0404bd-4fa8-4572-8f69-5f4d55c7397a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.860003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa681c8d-db26-48c8-a53e-ce9d7fac83e0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.876948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.877578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.877857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.878122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] PATH_INFO: `/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.879473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8ffdfac9ab4b53a043e4aa806e5446)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.881547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.886965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 222/448] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/projects/8d8ffdfac9ab4b53a043e4aa806e5446 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.888805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.904683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.905305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.905616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.905868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.906530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.906903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.908263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.917671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 227/449] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.923431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.948460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.000837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191fdca-5701-4810-ae5f-2ccc1c2c79c9 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.027819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2352e-ca51-4c31-8162-115a5309093a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.033703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.034324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.034614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.034929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] PATH_INFO: `/v3/groups/3970acbe616c460d952bd33f73f19817` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.036552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] RBAC: Authorizing `identity:get_group(group_id=3970acbe616c460d952bd33f73f19817)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.037988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.039598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 223/450] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/groups/3970acbe616c460d952bd33f73f19817 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:39.045282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.063702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.064312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.064593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.064865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.073215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.134590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.136172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.147889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8ba8-3212-4491-b6a7-3c67b4534b44 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.156273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.182737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.183364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.183639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.183910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.184626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] RBAC: Adding query filter params (group.id=3970acbe616c460d952bd33f73f19817, role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:39.184934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.186338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.191996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 224/451] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/role_assignments?group.id=3970acbe616c460d952bd33f73f19817&scope.project.id=8d8ffdfac9ab4b53a043e4aa806e5446&role.id=238ba40fcefe4923bb27057b1074fbef => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:39.239790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 228/452] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:45:39.245524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.343410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.354528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212c33d-407e-4cd2-b477-d194b1d9ff1d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.389973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.390596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.390934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.391217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] PATH_INFO: `/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.392342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] RBAC: Authorizing `identity:get_project(project_id=bbe873ec2e4b4dc7b12dd87b247ac3dc)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.393766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.396085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 225/453] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/projects/bbe873ec2e4b4dc7b12dd87b247ac3dc => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:39.402382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.505131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435104-f48a-4bc2-bfe4-6a516151850b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.525786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.540850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9016349-22be-41cb-85e2-a6c3573c4ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.541488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9016349-22be-41cb-85e2-a6c3573c4ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.541770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9016349-22be-41cb-85e2-a6c3573c4ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.542052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9016349-22be-41cb-85e2-a6c3573c4ea8 None admin] PATH_INFO: `/v3/groups/e9577aa276904d5aa438f8ac83528388` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.543200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9016349-22be-41cb-85e2-a6c3573c4ea8 None admin] RBAC: Authorizing `identity:get_group(group_id=e9577aa276904d5aa438f8ac83528388)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.544651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9016349-22be-41cb-85e2-a6c3573c4ea8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.546254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 229/454] 10.222.0.72 () {56 vars in 1180 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/groups/e9577aa276904d5aa438f8ac83528388 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:39.553037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.651930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d564e5-1723-47fb-a72c-77b974e60d8a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.684467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.685064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.685347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.685621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.686342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] RBAC: Adding query filter params (group.id=e9577aa276904d5aa438f8ac83528388, role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=bbe873ec2e4b4dc7b12dd87b247ac3dc) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:39.686653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.688105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19212c30-11a7-4c7f-8b6e-8acf05ba8bb9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.694571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 226/455] 10.222.0.72 () {56 vars in 1376 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/role_assignments?group.id=e9577aa276904d5aa438f8ac83528388&scope.project.id=bbe873ec2e4b4dc7b12dd87b247ac3dc&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:42.535661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cd3d5-ffc0-4f3d-88c7-9315afc46898 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.537273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cd3d5-ffc0-4f3d-88c7-9315afc46898 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.537778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cd3d5-ffc0-4f3d-88c7-9315afc46898 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.539359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 230/456] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:42.544515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9669-b01e-43eb-a31f-be9cb7018e17 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.545352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9669-b01e-43eb-a31f-be9cb7018e17 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.545822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9669-b01e-43eb-a31f-be9cb7018e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.556032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f03be-d2ea-4812-b6dd-b0bded7be258 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.556604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f03be-d2ea-4812-b6dd-b0bded7be258 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.557075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f03be-d2ea-4812-b6dd-b0bded7be258 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.558876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 231/457] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:42.564264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61826-2760-4c98-9d31-87337d179280 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.564816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61826-2760-4c98-9d31-87337d179280 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.565283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61826-2760-4c98-9d31-87337d179280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.575859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a3ef9669-b01e-43eb-a31f-be9cb7018e17 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:42.577080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a3ef9669-b01e-43eb-a31f-be9cb7018e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:42.599176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:42.603854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0ec61826-2760-4c98-9d31-87337d179280 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:42.605056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0ec61826-2760-4c98-9d31-87337d179280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:42.678522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 227/458] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:42.714220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 232/459] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:42.777764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68bbf-8452-4718-a188-a05a3edaa775 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.778392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68bbf-8452-4718-a188-a05a3edaa775 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.778923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf68bbf-8452-4718-a188-a05a3edaa775 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.782421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 228/460] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:42.788174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.788396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.788841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.812996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:42.814521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:42.907290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 233/461] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:43.040708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68bbf-8452-4718-a188-a05a3edaa775 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.043242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.128636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.226944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd1d2d-3e6d-41a9-a9b6-bb35c98a7517 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.229934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf68bbf-8452-4718-a188-a05a3edaa775 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.273343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.274088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.274369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.274639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.275444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.276078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.276355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.276625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.288245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.289235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.290104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.290658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:43.306058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:43.307006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:43.308035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:43.308035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:43.308035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-d5cc77c4-8d2c-42a3-9db0-91ce391399ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:43.308594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:43.309614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:43.310622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:43.310622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:43.310622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:43.310622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 229/462] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:43.311147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a8005-f19d-41b4-b400-ed4ec23d96f7 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.311430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a8005-f19d-41b4-b400-ed4ec23d96f7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.311699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a8005-f19d-41b4-b400-ed4ec23d96f7 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.312914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 230/463] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:43.313740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a8005-f19d-41b4-b400-ed4ec23d96f7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.314965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 234/464] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:43.315990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.321079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.425146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a8005-f19d-41b4-b400-ed4ec23d96f7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.433449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.436771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d4000-fa22-4f54-ab08-6c610629e718 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.465118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.465757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.466040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.466323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.473000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.474132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.474412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.474964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.475347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:43.475650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.477031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.478381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.479918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.490597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 235/465] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:43.491656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-d4cb2b36-e1e0-4406-9b31-d6f7d3c8b557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:43.495543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:43.496434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 231/466] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.502276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.517828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:43.519093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:43.578968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.607019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 232/467] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:43.608536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.632198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2ee43-51b7-477e-9132-f011116369c3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.670482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.673568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.674446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.674954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.675780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:43.676370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.679209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.695581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 236/468] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:43.697385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.728573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.744543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371170a3-be8c-4c19-89c9-ec50663928ae None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.788422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.789417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.789922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.790421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.791274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.792959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.823609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 233/469] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:43 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:43.825874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.833648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3125-c8de-4056-b469-ef87319036f7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.897537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.898169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.898528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.898807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.899558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:43.899864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.902873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.907726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.936094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 237/470] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:43.937292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:43.992390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f4af-1013-4615-8ccc-f511c1e590ad None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.030095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5745f-ee3d-4fee-8b6a-68d124bb180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.030720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5745f-ee3d-4fee-8b6a-68d124bb180a None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.031033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5745f-ee3d-4fee-8b6a-68d124bb180a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.031306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5745f-ee3d-4fee-8b6a-68d124bb180a None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.031963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5745f-ee3d-4fee-8b6a-68d124bb180a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.033413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5745f-ee3d-4fee-8b6a-68d124bb180a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.040037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.057611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 234/471] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:43 2022] POST /identity/v3/users => generated 246 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:44.063948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802340-9937-457b-b586-3e1c656e545c None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.063948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802340-9937-457b-b586-3e1c656e545c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.063948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9802340-9937-457b-b586-3e1c656e545c None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.063948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 235/472] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:44.063948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802340-9937-457b-b586-3e1c656e545c None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.069898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e468e3-5581-44e9-a520-453b1618aeca None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.117831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.118515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.118796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.119464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.131440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.132941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b913cbbb-340b-41cf-a700-8d40ca3b17a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:44.153524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:44.154510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:44.156075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:44.156075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:44.156075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:44.156075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 238/473] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:44.156075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.156363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.156539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.176503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:44.178121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:44.217088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.225553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9802340-9937-457b-b586-3e1c656e545c None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.260826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 239/474] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:44.261871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.270762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.271669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.272152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.272595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.273968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.276349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.308563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 236/475] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:44 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:44.340527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.388358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78486-4305-4dc6-9bfb-a46cb07333c0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.438425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.439474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.440002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.440542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.441453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:44.442544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.444548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.459684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 240/476] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:44.464925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.590763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.600995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e773c2-c98f-4cff-b730-2d1da9e37eab None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.664634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.665868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.666145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.666414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.667102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.668531 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.699594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 237/477] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:44 2022] POST /identity/v3/users => generated 249 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:44.741274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88a62-41b8-4eac-a8d4-ef3f0962a2d0 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.767221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.781311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.782023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.782369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.782709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.783493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.785054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.794993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.sql.core [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 17:45:44.794993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 17:45:44.794993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [parameters: {'id': '67c77cf0a03145b5a481df724fe820fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 17:45:44.794993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81410) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 26 17:45:44.813514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return ref.to_dict() Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:45:44.816114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application yield resource Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise self.value Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application session.commit() Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 17:45:44.818671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.flush() Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application compat.raise_( Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 17:45:44.820124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:45:44.821555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:45:44.827689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [parameters: {'id': '67c77cf0a03145b5a481df724fe820fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:44.829628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 17:45:44.831606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:44.834399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 241/478] 10.222.0.72 () {60 vars in 1131 bytes} [Mon Sep 26 17:45:44 2022] POST /identity/v3/roles => generated 144 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:44.834535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.837732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.984891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ae73d-8dda-43dc-bed5-e706b524390b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.023218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1ad98-9536-479a-a385-6ee771113794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.024150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1ad98-9536-479a-a385-6ee771113794 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.024610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1ad98-9536-479a-a385-6ee771113794 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.025094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1ad98-9536-479a-a385-6ee771113794 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.036446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1ad98-9536-479a-a385-6ee771113794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-48f1ad98-9536-479a-a385-6ee771113794 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:45.042279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:45.043210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:45:45.044002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:45.046201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 238/479] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:45.052445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.185384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c3140-f6d7-4107-abe9-ec1d4c00418a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.226103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.226718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.227019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.227297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.227960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:45.228259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.229636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94619390-a11f-4243-a37d-05901a5396cb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.245259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 242/480] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:46.554382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1904e-e8b8-4944-905e-7ab7b87adee8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.555105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1904e-e8b8-4944-905e-7ab7b87adee8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.555105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1904e-e8b8-4944-905e-7ab7b87adee8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.556614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 239/481] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:46.562154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f5cf2-61f5-4ed1-9c3c-dae2c4369514 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.562709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f5cf2-61f5-4ed1-9c3c-dae2c4369514 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.562709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f5cf2-61f5-4ed1-9c3c-dae2c4369514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.586920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-750f5cf2-61f5-4ed1-9c3c-dae2c4369514 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:46.588318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-750f5cf2-61f5-4ed1-9c3c-dae2c4369514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:46.654021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:46.685320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 243/482] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:46.888854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bb194-ec79-47da-a3d5-219ada62f922 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.889188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bb194-ec79-47da-a3d5-219ada62f922 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.889355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bb194-ec79-47da-a3d5-219ada62f922 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.891025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 240/483] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:46.896939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f5da2-022a-40bd-ac9f-84ab56d4da88 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.897232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f5da2-022a-40bd-ac9f-84ab56d4da88 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.897515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f5da2-022a-40bd-ac9f-84ab56d4da88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.919130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-291f5da2-022a-40bd-ac9f-84ab56d4da88 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:46.920319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-291f5da2-022a-40bd-ac9f-84ab56d4da88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:47.006611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 244/484] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:47.105963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bb194-ec79-47da-a3d5-219ada62f922 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.218059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.268488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bb194-ec79-47da-a3d5-219ada62f922 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.313728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.314027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a2195-12e0-4ef1-ac9c-1dfa1d3a6a20 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.314257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a2195-12e0-4ef1-ac9c-1dfa1d3a6a20 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.314364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.314536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a2195-12e0-4ef1-ac9c-1dfa1d3a6a20 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.314630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.314899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.316344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 245/485] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:47.323030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.323398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.323681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.327212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:47.332102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:47.333688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:47.335086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 241/486] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:47.339725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.364997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:47.366209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:47.446531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 246/487] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:47.501519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9b2f-baec-48ca-b52d-6fda6c7c4951 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.530661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.537613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.537811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.538613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.539302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.539581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.540929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:47.541232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:47.542652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:47.553696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 242/488] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:47.559236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.632061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.705646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530cf10-ed34-4973-8a58-81744fecccb3 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.772075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17469a-6958-4973-857f-f1856ffa10de None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.776772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.777417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.777696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.778007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.791808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:47.793514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-fbdf955a-f6ff-4ed1-bf0c-0532813d7430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:47.811177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:47.812128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:47.812957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:47.812957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:47.812957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:47.814939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 243/489] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:47.815813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.816220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df244688-a5ef-4766-a384-1b13b0c305b1 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.816515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df244688-a5ef-4766-a384-1b13b0c305b1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.816837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df244688-a5ef-4766-a384-1b13b0c305b1 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.817378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.817662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.817938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.818486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 244/490] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:47.819324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df244688-a5ef-4766-a384-1b13b0c305b1 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.833659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:47.838511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:47.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:47.840356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 247/491] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:47.842020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.880741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.939222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df244688-a5ef-4766-a384-1b13b0c305b1 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.950341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.955485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847f886-d424-4a94-846c-aeb51fb1da94 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.982776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a73f47-2016-42a9-9c9d-10b3e2ab404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.984004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73f47-2016-42a9-9c9d-10b3e2ab404d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.984004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73f47-2016-42a9-9c9d-10b3e2ab404d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.984151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73f47-2016-42a9-9c9d-10b3e2ab404d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.996559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a73f47-2016-42a9-9c9d-10b3e2ab404d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-f9a73f47-2016-42a9-9c9d-10b3e2ab404d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:48.002093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:48.003081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:48.003877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:48.004821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.004821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.004821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.004821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.005068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.005068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.006689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.008311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 245/492] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:48.009858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.009858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.010123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.024744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 248/493] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:48.024744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.030301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:48.031555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:48.104839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 246/494] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:48.105519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.139214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.143905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7dcf55-2768-441d-bb95-c19ead416856 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.182241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.182874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.183156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.183425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.184076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.184383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.185748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.194911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 249/495] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:48.195758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.218876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.223505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a84257-ed7f-4949-96ed-33bdac1e50ea None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.259576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.260179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.260470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.260718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.261367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.261667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.263039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad94f9-a3a4-41e6-8b5f-4cfc82edf453 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.276450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 247/496] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:48.276450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a97ff-e80a-4333-bd71-019b51e5009f None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.276450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a97ff-e80a-4333-bd71-019b51e5009f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.276450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a97ff-e80a-4333-bd71-019b51e5009f None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.276450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 248/497] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:48.276450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a97ff-e80a-4333-bd71-019b51e5009f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.307584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a374f-a894-47e0-9fd3-05d8a949510c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.345173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.345802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.346082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.346355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.355846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.357294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:48.370050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:48.371002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:48.371885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 250/498] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:48.373260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.390723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.399442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a97ff-e80a-4333-bd71-019b51e5009f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.433075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.433704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.433985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.434258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.444771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.446213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-d4835ced-63f2-4672-87e6-d9ff0ae26d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:48.461690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:48.462726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:48.463698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:48.463698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:48.463698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:48.463901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.463985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 249/499] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:48.464540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.464833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.465109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.483873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:48.485086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:48.488005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb742ef9-8ee7-4962-a843-5b4230c5603d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.523530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.524172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.524456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.524733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.535446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.537077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:48.551828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:48.552725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:48.553584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:48.553584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:48.553584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:48.553584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 251/500] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:48.553837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.574876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 250/501] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:48.574876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.640845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.660202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efeb50-f5a7-49da-8aa2-f3108424d9ed None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.665060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24dbec-6fcb-4ed1-8a49-390acfa1c083 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.689863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.696686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.697296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.697908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.698198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.698939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.699254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.700687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.704254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.704882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.705163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.705439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.716478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:48.721478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:48.722411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:48.723782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 251/502] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:48.727228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.742679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 252/503] 10.222.0.72 () {56 vars in 1137 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:48.743844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.844067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f81ed3-48eb-4092-809d-48785faa9d34 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.853645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001ee25-5306-4ddb-9066-8ac081cb39d8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.878288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.878917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.879699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.880436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.881075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.881376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.882743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.887356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.887957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.888234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.888502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.889150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.889448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.889778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.890804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.901118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 252/504] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:48.901118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.906528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 253/505] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:48.906528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.955854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.998203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1eef7-3703-46bb-a2f1-00066eb84d8a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.006238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109b1c-3806-4573-8b61-9bfee67526f1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.032462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.033064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.033341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.033617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.034266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:49.034567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.036107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.040382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.040992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.041279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.041553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.043876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 253/506] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:49.044659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.107210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.109410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.110889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.149733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf30b36-e1cb-4f16-aae4-cdddeba89230 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.158878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.183459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.184075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.184354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.184630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.195655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.197382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.210277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 254/507] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:49.211142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:49.212210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:49.213091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:49.213948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:49.213948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:49.213948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:49.213948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 254/508] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:49.214171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.315441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ed40d-e439-4923-b321-cc11c56549e8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.315687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52002f34-f1a4-41a5-a831-9e4554283e44 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.328860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.351208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.351738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.351995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.352281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.352371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.352558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.352647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.352883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.359512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.360313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.360906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.361717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:49.372562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:49.373483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:49.374545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 255/509] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:49.375331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:49.381374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:45:49.382529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:49.383430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.383515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 255/510] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:49.383515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.392499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.479000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda81ab-6b82-4e40-a5eb-308501de5c48 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.482719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af963b0b-0175-4f26-9222-b91004598c9c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.513121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.513778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.514062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.514334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.516030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.516664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.516959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.517232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.525275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.526710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.528325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.529765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:49.541930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:49.543037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:49.544119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:49.544119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:49.544119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:49.544119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 256/511] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:49.544119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:49.545704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:49.546549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:49.547411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:49.547411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:49.547411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:49.547411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 256/512] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:49.548019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.572093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.607843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.647789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f925180-8c3b-4dad-9884-d2bea26c05cd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.648110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c1c2-1458-4355-8671-84e048957217 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.681151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.681756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.682040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.682317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.682416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.683002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.683114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:49.683287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.683385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.683563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.684205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:49.684513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.684782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.685854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.698698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 257/513] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:49.699543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.703527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 257/514] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:49.704343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.785150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.800876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059e231-4631-40b7-9d6c-6a538d751303 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.804753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf739e45-2c5a-4428-8b5e-a76dec6f3e3b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.825728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.838028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.838630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.838944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.839215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.839934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:49.840237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.840336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.840902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.841188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.841461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.841773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.842098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:49.842434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.843821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.856091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 258/515] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:49.856892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.878035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 258/516] 10.222.0.72 () {56 vars in 1137 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:49.878861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.966410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e008c-986a-42ae-b7b1-c6808d5c2b65 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.991232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556c561-9546-42cc-bffc-bf0c9cc5c510 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.006730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.007361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.007642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.007915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.008549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:50.008852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.010218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.022242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 259/517] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:50.023053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.026887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.027412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.027695 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.027994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.028639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.030085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.047655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.066870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 259/518] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:49 2022] POST /identity/v3/users => generated 247 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:50.066870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.067174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.138715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec071c1-6a6f-46ce-8830-a964d7a83bb3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.181082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.181713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.181993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.182266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.183482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77df869-498d-4d09-aace-13b522cbfedf None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.224417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.224964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.225244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.225520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.232666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.234125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:50.247622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:50.248475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:50.249324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 260/519] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:50.249856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.268013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.269474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.318589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.321354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.363179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304703e-860e-4397-95d9-cdc1e07d6977 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.385685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 260/520] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:50.386561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.408349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.409311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.409795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.410274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.491369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03c9e1-7eca-4f0f-bd30-6e4890bd58ff None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.494201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.495475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.526780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.527390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.527674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.527950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.536507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.537975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.541600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:50.555872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:50.557069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:50.558017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 261/521] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:50.558957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.568419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.618439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 261/522] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:50.619888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.684456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776f4e4-1c1f-4b32-907c-0f2115652640 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.724781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.725395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.725678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.725955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.726640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:50.726984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.728424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.731205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a335e5-7d5e-408a-8cdd-f129219ad317 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.751675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 262/523] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:50.752301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.768158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.768789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.769073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.769352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.780613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.782079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:50.800568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:50.801579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:50.802527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:50.802527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:50.802527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:50.802527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 262/524] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:50.803363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.817478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.822207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.867958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce784801-b1a9-4397-8677-408cb895cc6c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.906377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.907023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.907366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.907647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.908371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:50.908681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.910117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.919184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38495586-10bb-492f-bbfa-56fb7fc1ab9f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.933509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 263/525] 10.222.0.72 () {56 vars in 1139 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:50.935106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.963592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.964227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.964502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.964763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.981944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.981944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:50.999332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.000314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:51.001324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:51.001324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.001324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:51.005487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 263/526] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.006379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.059875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c19dd8-7833-4dd8-bd44-d05bb3b46f66 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.072879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.078798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.096141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.096740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.097021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.097296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.098576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=10f80305dfe04303a9a9387535ed8ae4) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.098943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.100375 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.105787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 264/527] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/role_assignments?user.id=10f80305dfe04303a9a9387535ed8ae4&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:51.106567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.130482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ba382-daff-472e-be57-619cf65deaba None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.172568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.173256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.173541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.173817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.174482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.174798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.177031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.199586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 264/528] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:51.199586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.242560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628c289-9401-4655-a5a1-40741ccf2682 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.293762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.294679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.295185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.295636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.347747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419fa73-7743-4267-8e89-14f737e2992d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.364106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.378690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.386360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.387958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.393813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.394510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.394964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.395370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.396347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.396725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.398537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.413533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 265/529] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:51.414564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.498895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 265/530] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:51.499877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.542283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fae6c-5af3-4855-8667-c58e5ba2339e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.584063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.584647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.584930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.585206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.587810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.594421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.596714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:51.610895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.612704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:51.614591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:51.615454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c9df3-a1ee-4036-b61b-e8472c21d217 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.615586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 266/531] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.615586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.651060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.651979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.652324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.652667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.660123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.662221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.665743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:51.676893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.678827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 266/532] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.680417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.746198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59911fac-4922-4a02-90b0-768a220f0a00 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.791824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.792607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.793012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.793288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.795290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e6751-6018-4448-9e6c-5c5ab0e2c15d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.807159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.808622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:51.825458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.826377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:51.827844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:51.827844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.827844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:51.827844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 267/533] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.827844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.839796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.840785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.841319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.841789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.842732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.843277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.844974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.858871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 267/534] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:51.860168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.863416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.939742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.948332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c26c3-054a-4b8b-9862-55239e95be25 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.978497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05396b-fceb-4f19-90af-77c1f67d4e33 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.993155 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.993898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.994298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.994581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.995318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.995629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.997563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.018961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 268/535] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:52.019904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.020399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.021406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.021888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.022368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.023295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.023957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.025990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.040287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 268/536] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:52.041641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.110587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:52.143035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58958673-6190-449f-8d59-83a2832f7165 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.158700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524486c7-5a14-4f77-97d1-57bd959e3dde None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.187910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.188609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.188905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.189191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.190012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1adfb9b41a1d4d31bccf5cc87b955526) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.190330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.192022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.196760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.197654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.198276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 269/537] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/role_assignments?user.id=1adfb9b41a1d4d31bccf5cc87b955526&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:52.198822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.199090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.199660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.200653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=39bde61b0b7c4cf3950ef63248a3b0da) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.201174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.202874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41ad34-9e68-405f-9a5e-7e9ed10529aa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.210536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 269/538] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/role_assignments?user.id=39bde61b0b7c4cf3950ef63248a3b0da&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:52.224084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:52.325259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8088a-9ac6-4a21-85a5-3a6733514ccb None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.364946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.365514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.365791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.366059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.373913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.375423 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.378962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01d839-c6bf-4fee-9aa2-e094303fe0e8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.379324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01d839-c6bf-4fee-9aa2-e094303fe0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.379574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01d839-c6bf-4fee-9aa2-e094303fe0e8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.381299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 270/539] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:52.387515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.387832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.388070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:52.388280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:52.389075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:52.390214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 270/540] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:52.397330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.427937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:52.429192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:52.523841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:52.540350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 271/541] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:52.573887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b01592-9aae-4415-9d67-3533d1f2fc57 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.631891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.632435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.632717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.632990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.633691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.634002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.635472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.652633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 271/542] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:52.658574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.755463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:52.824514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3838f-74ad-49a3-aa31-d3d3808ac08c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.872211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.872822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.873101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.873372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.874093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=b179c391ce574cd8836b43ea5bd94222) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.874398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.875839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.887307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 272/543] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/role_assignments?user.id=b179c391ce574cd8836b43ea5bd94222&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:52.887560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.025139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b7dc2-f988-479c-81b1-6ec6917ecb07 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.071136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.071801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.072083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.072357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.086822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.091465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:53.096514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:53.100198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:53.101355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 272/544] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:53.104604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.231562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd69913-d3fd-41e3-9562-1620032f45af None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.252671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.269168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.269850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.270629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.270971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.271720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:53.272028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.273449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.282768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 273/545] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:53.291548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.406635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7f68-def8-4608-8df4-f55221cc6d8c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.450433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.451141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.451430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.451710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.463780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.465244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-e611e6e9-254d-42c4-9ee3-3153589fbc39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:53.486885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:53.489707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:53.491948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:53.491948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:53.491948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:53.492387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 273/546] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:53.497704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.644157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6f202-2932-4d4d-92bc-9966ad068212 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.670146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d58c8e-e68d-43f9-855e-ec101541d434 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.670480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d58c8e-e68d-43f9-855e-ec101541d434 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.670692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d58c8e-e68d-43f9-855e-ec101541d434 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.672075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 274/547] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:53.677798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.678082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.678360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.690210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5501d-743e-432a-a870-8c754602431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.690820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5501d-743e-432a-a870-8c754602431b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.691422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5501d-743e-432a-a870-8c754602431b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.691790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5501d-743e-432a-a870-8c754602431b None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.692925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5501d-743e-432a-a870-8c754602431b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:53.693231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5501d-743e-432a-a870-8c754602431b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.695850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5501d-743e-432a-a870-8c754602431b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.711032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 274/548] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:53.716001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5501d-743e-432a-a870-8c754602431b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.716205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:53.717337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:53.766497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.833913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5501d-743e-432a-a870-8c754602431b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.843542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.872999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.873641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.873923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.874194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.882529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.884390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:53.898378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:53.900603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:53.902774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 275/549] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:53.910992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.913147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 275/550] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:54.015693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937f579-69b7-422d-b546-0df65719c221 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.017251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:54.056195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:54.056859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.057152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.057430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.058163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:54.058472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:54.059945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:54.083728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 276/551] 10.222.0.72 () {56 vars in 1135 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:54.088967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:54.259433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856e689-6b47-41f9-bd84-edfed56523f0 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.295296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:54.295832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.296118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.296391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.313540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:54.361951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:54.384580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:54.386138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:54.478707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c5052-8f1f-43a8-a101-b50485c50fd4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.503170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 276/552] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:54.511937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:54.545960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:54.546522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.547655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.547933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.569151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:54.576615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:54.578556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:54.579409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:54.581806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 277/553] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:54.588556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:54.627479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:54.675430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:54.723373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8455c9-41e1-40ef-a17f-53fcf03b9e88 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.780037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:54.780689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.780970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.781246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.793294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08237ca-85ec-4c06-ba4b-0705878e68ff None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.808086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:54.809480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-04194907-f37c-4f68-8b44-fb20f70414bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:54.831509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:54.833182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:54.834054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:54.834054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:54.834054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:54.834054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 277/554] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:54.839199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2cf4d-ee47-45f1-b5c4-85f5c451f5cd None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.839199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2cf4d-ee47-45f1-b5c4-85f5c451f5cd None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.839199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2cf4d-ee47-45f1-b5c4-85f5c451f5cd None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.839199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 278/555] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:54.841129 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2cf4d-ee47-45f1-b5c4-85f5c451f5cd None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:54.853894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:54.854565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.854798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.855112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.855789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:54.856098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:54.857468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe74cd-ac86-43b9-9b45-1cbe3abc89a2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:54.866865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 278/556] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:54.868488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.868943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.869229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.893145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:54.895722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:54.952898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2cf4d-ee47-45f1-b5c4-85f5c451f5cd None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.987509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 279/557] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:54.989214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.003354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.006851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.021984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.022263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.022549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.022824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.023533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.023924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.025281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.049177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 279/558] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.060239 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.150422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6ecbb-a54e-4698-8f7e-a474cdb55b05 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.188842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.189530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.189813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.190083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.201064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.202507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.210220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccd242-1bed-410d-8cb4-099b146fc490 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c67eec99-bdc9-4cb4-865f-62b157c8e5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:55.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.219635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:55.220552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:55.220552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:55.220552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:55.220552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 280/559] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.221672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20591b-e1c2-4849-bca5-1fba20aef0eb None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.222108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20591b-e1c2-4849-bca5-1fba20aef0eb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.222436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20591b-e1c2-4849-bca5-1fba20aef0eb None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.225060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 281/560] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:55.229586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20591b-e1c2-4849-bca5-1fba20aef0eb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.254125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.254786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.255110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.255663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.264667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.266158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a1b252db-1f78-4fc0-890d-8c9816efd514 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:55.285189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.287741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:55.290236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:55.298079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 280/561] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.298079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.298079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.298079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.301560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.318826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:55.320997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:55.345153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20591b-e1c2-4849-bca5-1fba20aef0eb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.377121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.382306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.382938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.383267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.383538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.384169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.384517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.385940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.406879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 281/562] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:55.406879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.407142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 282/563] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.407142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.520564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f46b0-faeb-4aa3-aa32-ad8d917951f8 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.527076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf698922-dcc8-4e6f-ba71-fedcd85e3422 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.547521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.560858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.561655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.562041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.562430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.572924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.573809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.574235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.574653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.575917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.576182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:55.576547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.578170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:55.581657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.583084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:55.584216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:55.586168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 283/564] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.587244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.598626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 282/565] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.599849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.667333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.703694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bbfa4-fc40-417c-ac19-59a055460e90 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.732119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79995d3a-2e03-411a-b122-ea6f88a128cf None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.737996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.738677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.739004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.739280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.746521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.748015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:55.760295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.761270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 284/566] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.762267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.769474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.770760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.771929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.771929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.773376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.773686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.775089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.783208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 283/567] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.783992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.811750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.866111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4d85b-69ab-4b54-beb0-700ddc0bb763 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.885372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a803-6464-4008-84b8-d852cf04269a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.887810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.898926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.899527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.899802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.900071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.900782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=d2ba853553814d49bf3c3a92c29fd27f) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.901086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.902521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015774e-9a86-47e0-b6d4-8cf15850b074 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.907840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 285/568] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/role_assignments?user.id=d2ba853553814d49bf3c3a92c29fd27f&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.908783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212bf7f-5bc3-4f80-8d23-a53a1ffb4676 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.909066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212bf7f-5bc3-4f80-8d23-a53a1ffb4676 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.909339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212bf7f-5bc3-4f80-8d23-a53a1ffb4676 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.910877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 286/569] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:55.911502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212bf7f-5bc3-4f80-8d23-a53a1ffb4676 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.936969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.937741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.938029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.938307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.951109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:55.956525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.957985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:55.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:55.963374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 284/570] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.964570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.047861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212bf7f-5bc3-4f80-8d23-a53a1ffb4676 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.058493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.086161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.087097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.087097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.087340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.088045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.088350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.089785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3564edf-e09d-4c71-a715-77a0a93da298 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.098761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b043e-1547-4e23-8b4a-351bd778bd96 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.114042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 287/571] 10.222.0.72 () {56 vars in 1137 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:56.114042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.114042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.114042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.130009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:56.131247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:56.139530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.140265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.140615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.140956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.152558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.154084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:56.169141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.170353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:56.171630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:56.171630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.171630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:56.177739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 285/572] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:56.178466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.197374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.218865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 288/573] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:56.218865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.300372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120fffb-d15f-431d-a846-db056208e73f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.337034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.342914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.343430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a4bd2-8568-4fe1-9cb2-85c91ac94545 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.343968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.344428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.344904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.345744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.346235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.347820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.358890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 286/574] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:56.360141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.380714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.381332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.381612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.381948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.393055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.394565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:56.411413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.412590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:56.413700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:56.413700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.413700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:56.417277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 289/575] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:56.418187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.462988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.490095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eda33b-a76d-4ee2-b33e-feae8df70413 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.530447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.531089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.531369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.531641 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.532280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.532582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.533936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.544118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bacdd0-6087-413c-817e-fa0c842ef051 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.545809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 287/576] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:56.546631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.584585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.585503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.585787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.586066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.597565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.599714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.608597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:56.615368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.616513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:56.617725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:56.617725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.617725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:56.617725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 290/577] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:56.617725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.666928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a6a57-d56a-44ec-9bbb-cc65f17cec63 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.704461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.705451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.705451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.705835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.706554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.706979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.708466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.715823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.722223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 288/578] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:56.723041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.736495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312196d-9515-4b7d-8868-90edf06ce735 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.771745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.772381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.772662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.772938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.783869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:56.789497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.790660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:56.791577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 291/579] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:56.792295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.841285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6907c-c04c-403b-aa6e-c1896e5fae80 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.856509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.877516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.878369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.878648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.878952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.888612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.890056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:56.901541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.902477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:56.903464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:56.904021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88762dd7-cbad-4668-b365-df8716f885a3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.904515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 289/580] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:56.904515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.941459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.942588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.942939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.943162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.943801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.944357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.945769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.962274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 292/581] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:56.962274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.974243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.033278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089e6a7-ac10-437b-9fcd-54a9e68c1aa0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.069691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.070558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.070998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.071339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.074637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561b352-3a6c-466d-b22b-b92b2507966f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.078617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.080214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:57.091972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:57.093016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 290/582] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:57.093929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.103036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.120030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.120647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.120937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.121217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.121888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:57.122213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.123625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.134869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 293/583] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:57.134869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.205801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340e2ba-3b79-4741-8d9b-35335102e9c6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.223347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.241940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.242544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.242829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.243142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.243862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:57.244173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.245598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.251420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407c9df-3eb5-46b4-a6f4-46c11f2e60b7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.263430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 291/584] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:57.264255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.297646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.298820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.299341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.299796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.311168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.312698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:57.326505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:57.327726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:45:57.328896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:57.329674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 294/585] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:57.330360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.382979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.396510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9c91f-6d75-432d-98e4-c77fbb2ab70e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.433398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.434260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.434713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.435033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.436145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:57.436396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.437850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.444907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0b511-9f72-48ba-be01-5e5843e8d72c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.457528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 292/586] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:57.457528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.480014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.480584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.480864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.481137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.492401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.493846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.496192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:57.509566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:57.510738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:57.511868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:57.511868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:57.511868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:57.512432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 295/587] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:57.513039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.576400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f49b67-e08d-4ac9-9321-10da9aad9b96 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.612937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.614804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.615662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.616012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.616296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.620903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c3deb-8e85-48cd-8f98-9fe3475e5845 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.627700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.629258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:57.647797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:57.649067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:57.650177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:57.650177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:57.650177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:57.653579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 293/588] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:57.654507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.665417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.666083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.666355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.666670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.667423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:57.667732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.669203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.693390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 296/589] 10.222.0.72 () {56 vars in 1137 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:57.694549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.782432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.805860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fb9b0-03f0-42d0-839a-eb96e21a4fe7 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.818868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d164e08-6754-4b12-b88a-60cc163aab5a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.859722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.861566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.861566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.861566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/10f80305dfe04303a9a9387535ed8ae4/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.867055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.867766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.868113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.868456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.869166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:57.869536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.871014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.878115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=10f80305dfe04303a9a9387535ed8ae4, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.880630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.883685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 297/590] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:57.884728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.901327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.905206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 294/591] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:57 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/10f80305dfe04303a9a9387535ed8ae4/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:57.906196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.047844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a501e-8d92-4c03-9590-42977b6dddf1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.090283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.091024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.091308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.091586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.101252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:58.106152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.108744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.112820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d6c19-e77b-4edd-8a88-101a30552653 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:58.133945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:58.134979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:58.135893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:58.135893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:58.135893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:58.135893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 295/592] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:58.136784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.181360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.181982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.182260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.182529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.183203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:58.183503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.184878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88365d01-8a40-412b-9036-10603491bf62 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.202641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 298/593] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:58.203932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ae28d-ea62-48bd-a11e-877f94f24e61 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.204217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ae28d-ea62-48bd-a11e-877f94f24e61 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.204485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ae28d-ea62-48bd-a11e-877f94f24e61 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.205830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 299/594] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:58.211874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ae28d-ea62-48bd-a11e-877f94f24e61 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.255052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af691c4b-9553-451a-8e7d-9c5c6c95aff1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.289908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.290563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.290872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.291145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.292052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:58.299937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.302119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:58.314730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:58.315722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:58.317472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 296/595] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:58.318672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.330123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ae28d-ea62-48bd-a11e-877f94f24e61 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.373867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:58.375623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.376239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.376517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.376794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.377429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:58.377738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.379807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146d97e-6cb2-4c28-a7b8-eaf21cbcb996 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.391640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 300/596] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:58.393007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.393319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.394052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.414070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:58.415296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:58.434733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618f84-41a0-467c-9bf5-98caba3e1b96 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.470100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.470734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.471083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.471360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.479292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.480784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.491027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 301/597] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:58.491027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:58.496836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:58.497772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:58.498693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:58.502659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 297/598] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:58.503527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.571020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:58.626680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99ea7-bb04-4a85-b694-7627de140326 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.630400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fed0bd-4863-4bb3-948a-ea2dd71707af None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.642575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:58.675443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.676679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.676967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.677261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.677970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:58.678176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.678380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.679160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.679578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.679755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.680142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.692799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.694734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.704396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 302/599] 10.222.0.72 () {56 vars in 1139 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:58.705299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:58.709730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:58.710784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:58.711783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:58.718746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 298/600] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:58.718746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.827346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea84187-b922-4adb-a79b-c69838ad546f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.828558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650b206-7932-4a32-af5b-67eb1fcbe70e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:58.851089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:58.868535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.868959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:58.869141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.869425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.869696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.870207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:58.870373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:58.870679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.870957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:58.871366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:58.872164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.884001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:58.884591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 303/601] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:58.885386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.885976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:58.903960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:58.909866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:58.911056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:58.911056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:58.911056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:58.911056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 299/602] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:58.911056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:58.938124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.018412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d2bc7-aa94-4243-bd03-c1c28777be38 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.040718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf2ef9-7ca8-4918-881d-835f55078c74 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.055017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.055708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.056063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.056416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.057183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:59.057498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.058977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.072885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 304/603] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:59.073689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.085127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.086396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.086859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.087278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.099220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:59.104981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:59.106090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 300/604] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:59.106991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.123937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.187459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d074c-ccbe-4ef4-9888-f0dc8d880846 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.204766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.223941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.225110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.225554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.225833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1adfb9b41a1d4d31bccf5cc87b955526/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.227194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8c22d-e08e-4fda-9032-da5ec57c7e31 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.241071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=1adfb9b41a1d4d31bccf5cc87b955526, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.242525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.256741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 305/605] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:59 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1adfb9b41a1d4d31bccf5cc87b955526/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:59.257427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.291002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.291638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.291914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.292181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.292846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:59.293452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.294548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.310970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 301/606] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:59.310970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.400755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.415842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46933a1a-85f0-4b89-9a41-7462c3a73760 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.446403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6d313-4498-4887-9345-091fc2fa3fbf None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.467282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.468403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.469086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.469369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/39bde61b0b7c4cf3950ef63248a3b0da/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.487295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=39bde61b0b7c4cf3950ef63248a3b0da, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.488172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.488607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.489073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.489430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.489714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.490374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:59.490670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.492102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.501285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 302/607] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:59.502382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.506959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.511115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 306/608] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:59 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/39bde61b0b7c4cf3950ef63248a3b0da/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:59.511997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.710701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeede2-6ea8-4b6b-9b1b-bafbd8f5a919 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.744837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9cd8-41c6-470e-929d-c5f8c2790f31 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.752350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.753536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.753826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.754184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.767876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.772165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.774749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.790099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.790790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.791099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.791370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:59.795260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:59.796170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:59.797246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:59.797246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:59.797246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:45:59.797246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 307/609] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:59.798563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.800113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.805978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:59.818518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:59.819816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:45:59.820714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 303/610] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:59.825713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.891868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:59.934383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606674c7-fc44-4461-a0ea-ab724fc964da None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.934897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50783c9d-1f97-4bd8-bd58-272f98a58f5f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:59.970478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.971044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.971327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.971601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.972371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:59.972602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:59.973361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.974187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.974679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.974783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.975306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.976189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:59.977168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:59.978778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:59.988482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 304/611] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:59.994544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.997795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 308/612] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:00.003047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.077598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:00.100606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d1685-4a8f-4540-b595-316eff34f084 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.128696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:00.142177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b9b-9d57-48ee-bdd8-e20aed7bc11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.142884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b9b-9d57-48ee-bdd8-e20aed7bc11b None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.143215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b9b-9d57-48ee-bdd8-e20aed7bc11b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.143497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b9b-9d57-48ee-bdd8-e20aed7bc11b None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/b179c391ce574cd8836b43ea5bd94222/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.149579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2bdf-2d73-48ca-b39c-cb2521b5b634 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.158865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b9b-9d57-48ee-bdd8-e20aed7bc11b None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=b179c391ce574cd8836b43ea5bd94222, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.160931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b9b-9d57-48ee-bdd8-e20aed7bc11b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.193398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.194649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.195229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.195703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.197697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 305/613] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:59 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/b179c391ce574cd8836b43ea5bd94222/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:00.204529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.206429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:00.220307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:00.221362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:00.222350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:00.225847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 309/614] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:00.231521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.232041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff0bb9-72dc-4961-9d0f-baf15e29419b None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.232331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff0bb9-72dc-4961-9d0f-baf15e29419b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.232610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff0bb9-72dc-4961-9d0f-baf15e29419b None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.234289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 306/615] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:00.240377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.240706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.240989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.279926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:00.281158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:00.365192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a18373-422a-489c-ac8f-9a3facf687e1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.381415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:00.420738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.421825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.422313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.422785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.423760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:00.424268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.426726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.451182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 310/616] 10.222.0.72 () {56 vars in 1135 bytes} [Mon Sep 26 17:46:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:00.459866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.471180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 307/617] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:46:00.571030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f18492-7eb1-4b61-acf3-369c8c8173a3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.614145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.614793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.615103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.615379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.626339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.627819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:00.646710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:00.647727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:00.648612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:00.648612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:00.648612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:00.651911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 311/618] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:00.657638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.808512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbf961-22d6-4ebb-bd4a-a1541bbc0ed8 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.815926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:00.868371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.868905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.869178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.869456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.870110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:00.870418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.871817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.885142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.887016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 308/619] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:00.891517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.929083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:01.001345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e25b8-a528-420b-8384-1a54ee4ee4e2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.039956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.040582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.040863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.041132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.048887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:01.050821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:01.063730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:01.064672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:01.065559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:01.070364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 309/620] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:01.071599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:01.101166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:01.137063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6104329-6350-4ae7-9a40-f0fd45af86e6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.187101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e53f28-aca7-4159-bfd8-5e83cbc3336c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.228194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.228840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.229110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.229385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.242142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.242757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.243066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.243344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.244108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:01.244422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:01.245856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:01.252152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:01.259189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:01.260343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:01.261374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 312/621] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:01.266267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:01.273421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 310/622] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:01.279068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:01.357407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:01.381824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c911aa8-cd1e-4398-b09d-933b40f89418 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.440249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.440786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.441064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.441337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.442870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:01.443184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:01.445099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:01.463131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 313/623] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:01.470192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:01.497608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:01.538375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2932d3-676e-47f9-aa1f-6c1768c8ff0d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.614766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.617764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.618057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.618335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/d2ba853553814d49bf3c3a92c29fd27f/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.625953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7018f-b73e-4a24-9cd9-113a05986edb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.651672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=d2ba853553814d49bf3c3a92c29fd27f, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:01.653115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:01.678459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 311/624] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:01 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/d2ba853553814d49bf3c3a92c29fd27f/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:01.691229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.692401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.692684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.692953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.705545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:01.707236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:01.719334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:01.732040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:01.736365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:01.737861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:01.737861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:01.737861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:01.737861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 314/625] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:01.744989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:01.896658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a02d15-ff94-4fdd-8eeb-e45286d499c1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:01.939671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:01.940298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:01.940577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:01.940849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:01.942140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:01.942446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:01.944350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179a9a8-8cdc-457b-ad55-7a0d3bc54fe8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:01.946130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:01.961457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 312/626] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:01.968536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:02.083067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699790a-20bd-4dba-8661-f9509f619bc1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:02.087027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eacf4-8287-4091-8767-8e554555e463 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.087315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eacf4-8287-4091-8767-8e554555e463 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.087593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eacf4-8287-4091-8767-8e554555e463 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.089134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 313/627] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:02.094322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.094615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.094919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.123839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:02.124777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.125239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.125688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.134486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:02.136267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:02.143875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:02.145648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:02.150302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:02.151362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:46:02.152320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:02.154624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 315/628] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:02.162309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:02.280515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:02.308508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 314/629] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:46:02.351250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b9a1-cf9c-4cf5-9a0a-8b1c410f163e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:02.407719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:02.408344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.408629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.408908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.409653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:02.409964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:02.411976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2390e-4222-486e-a131-bff6f791c67f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:02.436529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 316/630] 10.222.0.72 () {56 vars in 1137 bytes} [Mon Sep 26 17:46:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:02.444211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:02.489526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2135c6-3836-4233-a7d6-1212da56f96d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.489867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2135c6-3836-4233-a7d6-1212da56f96d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.490079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2135c6-3836-4233-a7d6-1212da56f96d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.491968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 317/631] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:02.499060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.499347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.499865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.532808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:02.533931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:02.609263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91550201-11a2-4875-8a02-4aa5a8657781 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:02.639625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:02.655578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01cef4-872c-4e34-93ac-13433e133164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:02.656303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01cef4-872c-4e34-93ac-13433e133164 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.656587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01cef4-872c-4e34-93ac-13433e133164 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.656857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01cef4-872c-4e34-93ac-13433e133164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.699941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 318/632] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:46:02.740017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:02.761588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01cef4-872c-4e34-93ac-13433e133164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:02.766869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01cef4-872c-4e34-93ac-13433e133164 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:02.847392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:02.947465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e5844-28b0-4d19-96ed-cb045b74acbe None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:02.960538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 315/633] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:02.962213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591f6e-dd7f-4f27-b7fe-77e35ee30765 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.962803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591f6e-dd7f-4f27-b7fe-77e35ee30765 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.963304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591f6e-dd7f-4f27-b7fe-77e35ee30765 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.964862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 316/634] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:02.965892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591f6e-dd7f-4f27-b7fe-77e35ee30765 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:02.997802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab433610-d6a7-4d51-86ea-7b50cb7ae126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:02.998545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab433610-d6a7-4d51-86ea-7b50cb7ae126 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.998861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab433610-d6a7-4d51-86ea-7b50cb7ae126 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.999135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab433610-d6a7-4d51-86ea-7b50cb7ae126 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.999262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.009841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab433610-d6a7-4d51-86ea-7b50cb7ae126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ab433610-d6a7-4d51-86ea-7b50cb7ae126 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:03.014684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.015722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:03.016802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 319/635] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.017712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.017712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.017712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.036547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:03.037815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:03.093685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591f6e-dd7f-4f27-b7fe-77e35ee30765 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.105936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 320/636] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:03.106729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.138065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.139040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.139501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.139949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.148462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.152768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.155880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:03.173295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.174288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:03.175428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:03.175428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:03.175428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:03.178051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 317/637] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.179261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.218107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b6cee-82ed-4cef-9539-b118c1493b46 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.267439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.268078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.268359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.268632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.272634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.282621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:03.289054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.290258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 321/638] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.291484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.306165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a04ed-a559-4e5f-a9a0-8fa92dcbab11 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.348252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.349328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.349860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.350353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.351253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:03.351845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.354087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.365291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 318/639] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:03.366779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.407382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.415774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac650371-6bee-4ac6-bc85-7a49aca09be7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.453064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.453445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.453732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.454005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.454641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:03.454973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.456345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.468725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 322/640] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:03.469430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.489562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d60520-4532-401f-a479-34e3588ac7bd None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.527882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.528653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.528932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.529205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.529862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:03.530165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.531552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.537433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.546311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 319/641] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:03.547205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.578867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e3624-9d6e-41d5-9083-2bc8ed9b9690 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.614290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.614960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.615227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.615549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.628815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.630475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:03.647377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.648395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:03.649363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:03.649363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:03.649363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:03.649363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 323/642] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.649363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.662864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.663148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2f5f0-245c-4420-ab71-d7cd0e3ce01a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.703407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.704047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.704311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.704586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.715702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:03.720468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.721483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:03.722324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 320/643] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.723625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.764948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f18f2-01e7-4ccd-a983-904ce6307bb4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.788404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.800121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.800790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.801591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.801874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.808836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.810303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:03.822704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.823852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:03.825110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:03.828633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 324/644] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.829475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.839206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1e622-2ca1-4ec7-ac86-9d80838ad320 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.875162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.875717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.876052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.876331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.887877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.889445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.899287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:03.909174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:03.910236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:03.911846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:03.911846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:03.911846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:03.911846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 321/645] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:03.911846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.945321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643cff9-6a04-4cbf-98cd-2524d6486c88 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.994471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.995111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.995399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.995675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.996319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:03.996633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.998004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.010438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 325/646] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.011306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.035906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bab16-8b60-41cf-b89e-f7c8425826af None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.048917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.073953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.074737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.075052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.075327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.076050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.076356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.078087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.093889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 322/647] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.094716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.121654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906b974-dc7e-4db5-a496-b1c1007b7f8e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.147983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.158011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.158630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.158948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.159242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.159911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.160949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.162397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.174881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 326/648] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.175234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.212446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a08fc-4d7b-47e2-a047-259c6530cf6d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.247811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.248411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.248785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.249068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.249712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.250014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.251420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.264433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 323/649] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.265291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.292960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5875d4f-820b-47ef-8e95-63e3c38c407a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.300361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.327365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.327989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.328269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.328540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.336395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.337839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:04.349331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:04.350457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 327/650] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:04.351605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.371694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588cece-575c-4b1b-a2fe-f0b4ff0d4ec8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.397601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.405915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.406482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.406758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.407059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.418942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.420463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-960fdf2e-513a-4861-a57b-9a7e123b7629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:04.435510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:04.436490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:04.437928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:04.437928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:04.437928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:04.437928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 324/651] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:04.437928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d273f-2e48-4be2-82c3-48d1dd8d9c5a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.438289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d273f-2e48-4be2-82c3-48d1dd8d9c5a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.438289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d273f-2e48-4be2-82c3-48d1dd8d9c5a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.439514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 325/652] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:04.440205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d273f-2e48-4be2-82c3-48d1dd8d9c5a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.461080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a196bc7-cbdd-4619-91fc-18f1641cf8b4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.495654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.496255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.496540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.496812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.497534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=10f80305dfe04303a9a9387535ed8ae4) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.497846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.499444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.505402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 328/653] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/role_assignments?user.id=10f80305dfe04303a9a9387535ed8ae4&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.505402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.533686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.554010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d273f-2e48-4be2-82c3-48d1dd8d9c5a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.595870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.596533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.596818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.597097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.607591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.609063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:04.622617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:04.623679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:46:04.624650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:04.625193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f6c5-ba83-418a-9296-271de1631242 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.628812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 326/654] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:04.630453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.636708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.663986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.664675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.665014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.665308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.666023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.666328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.667825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef239-75ab-41e1-a383-7c342886609c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.686934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 329/655] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.688015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.688313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.688591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.710045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:04.711280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:04.754820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc7188-fc21-48e8-987c-917ba5520ac2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.790744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 330/656] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:04.791642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.795222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.795839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.796123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.796400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.797050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.797357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.799019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.817928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 327/657] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.819122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.829646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.920473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:04.924341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f19f2-9649-413b-9729-8f75e9138948 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.946959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42c397-b33e-4e4b-a2a7-90815c547843 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:04.963543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.964299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.964833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.965211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:04.966094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:04.966505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:04.968199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:04.992849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 331/658] 10.222.0.72 () {56 vars in 1137 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:04.993772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:04.998439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:04.999132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:04.999416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:04.999689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.084877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.086357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.094280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.104656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f6c0-ba0a-474d-904e-206481ee8b08 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.142826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.143477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.143764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.144035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.151815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.153245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:05.166020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:05.167036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:46:05.167976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:05.171343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 332/659] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:05.172453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.195211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 328/660] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:05.196313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.201709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.294060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8690c-446b-4f7d-86be-f95218bfec40 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.308344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba81f-47e0-403c-b24f-467141227628 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.317573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.331620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.332851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.333173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.333446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.348368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.349031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.349410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.349684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.360850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:05.367177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:05.368105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:05.368957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 329/661] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:05.369687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.412791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.414262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.433989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.480753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b52cf-2e7f-484f-a2b6-defbb9a832fb None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.520950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.521624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.521844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.522117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.522874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:05.523187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.524613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.529740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 333/662] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:05.529740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.550594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 330/663] 10.222.0.72 () {56 vars in 1139 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:05.550594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.581740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.637438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c96d9-879c-491c-8e96-abec41c48931 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.655259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.665339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56382e2-d5ee-412f-9bf4-cdd826574cda None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.676318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.677240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.677694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.678108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.690262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.692338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.704381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.705057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.705341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.705612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.706274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:05.706577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.708698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:05.709243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:05.710116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:05.711057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:05.711057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:05.711057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:05.711057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 334/664] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:05.711301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.727535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 331/665] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:05.727535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.816172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.823921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c23561-c300-4046-8c07-4e6143b0b87c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.837157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc076b99-c9a0-42af-bee7-ab7eeedf3007 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:05.859974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.860775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.860890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.861145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.872111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.873556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.875891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:05.876813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.877090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.877358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:05.888144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:05.889108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:05.890037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:05.890037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:05.890037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:05.890037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 335/666] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:05.890037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:05.914192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:05.960971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:05.962756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:05.998112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4a6d7-33ba-4eaf-ba5f-595b71d49bee None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.033441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.034035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.034473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.034751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.036012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:06.036318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.037682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.054116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 336/667] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.054116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.061002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.081793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 332/668] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:06.083261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.144230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.165897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52611f9b-c7ad-402f-bb3f-909a33de0a6e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.194426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ef208-5dd5-472b-a777-e19d307a717f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.202445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.203100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.203378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.203642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.214503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.215992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:06.230671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:06.231874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:06.232787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:06.232787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:06.232787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:06.232787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 337/669] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:06.233028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.233754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.234367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.234645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.234957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.235605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:06.235901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.237504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.250211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 333/670] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.250869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.302389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.348705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e2b33-3b22-46c9-8466-22e43486a44b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.360632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab4996-e633-4faa-8ce4-5f67d1e91fd8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.379773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.386359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.387153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.387501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.387843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.395048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.396534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.399539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.400249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.400603 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.400947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:06.408408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:06.409456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:06.410560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:06.416387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 338/671] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:06.416387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.418336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.419968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:06.437444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:06.438661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:06.439859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:06.439859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:06.439859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:06.440490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 334/672] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:06.441180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.528222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05519ea-2201-4dfc-bf37-98fe1f00c190 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.552266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.557469 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7112f5-94e5-4b40-ac86-d73cf4706961 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.564273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.564886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.565171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.565445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.566214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:06.566552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.568014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.585958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 339/673] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.585958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.602071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.602792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.603122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.603394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.610433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.612123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:06.624125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:06.625031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:06.625936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:06.629973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 335/674] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:06.630649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.640618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.692705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98e0f9-4049-45b6-81da-8c13f7148b0c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.734478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.735121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.735408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.735680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.736363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:06.736667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.738089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.744982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd58230-8cdb-443f-818f-fd303dbe2184 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.758247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 340/675] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.758247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.785148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.786179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.786719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.787025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.787667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:06.787976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.789323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.806537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 336/676] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.806537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.826654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.866140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb814a-77cd-4048-a9ea-7e901b44a95c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.879862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.902685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.903329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.903605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.903873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.908338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec4b77-1d4b-40b8-9882-614f5d563e1f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.911556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.912977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:06.923937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:06.925054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:06.926350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:06.930053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 341/677] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:06.930900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.945706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.946304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.946579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.946863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.947556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:06.947861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.949273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.961262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 337/678] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.962046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.042307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:07.043968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6425434-faec-4c37-a328-082659b7f8ab None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.063725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21bd39-1ff7-4f40-b373-204e6e4b4cd0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.077717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.078322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.078598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.078894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.079612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1adfb9b41a1d4d31bccf5cc87b955526) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:07.079919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.081378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bece812-5d0a-4e6d-97b9-0eecd4756c4d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.087712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 342/679] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/role_assignments?user.id=1adfb9b41a1d4d31bccf5cc87b955526&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:07.088865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c923d0c-c684-48f3-8358-89c83c5346f8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.089151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c923d0c-c684-48f3-8358-89c83c5346f8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.089426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c923d0c-c684-48f3-8358-89c83c5346f8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.095126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 343/680] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:07.095126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c923d0c-c684-48f3-8358-89c83c5346f8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.105793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.106455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.106738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.107054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.118519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.120041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:07.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:07.135841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:07.136829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 338/681] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:07.137764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.141624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:07.216268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c923d0c-c684-48f3-8358-89c83c5346f8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.256142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.256760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.257036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.257304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.258193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:07.258494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.259951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da163ab-4368-4ae2-8144-67528f593480 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.265910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4ec3a-cbe1-4c70-b842-916893e87997 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.282080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 344/682] 10.222.0.72 () {56 vars in 1135 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:07.283358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.283647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.283931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.305409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:07.307147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:07.313278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.313897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.314180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.314455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.315223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=39bde61b0b7c4cf3950ef63248a3b0da) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:07.315539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.317704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.327480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 339/683] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/role_assignments?user.id=39bde61b0b7c4cf3950ef63248a3b0da&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:07.328329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.331195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:07.415955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 345/684] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:07.417663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.452064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:07.465592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8a1f0-ba85-44b3-9e63-5f1ad114ce89 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.511317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.512085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.513110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.513517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.514456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:07.515018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.516701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.547108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 340/685] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:07.552967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.573614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9854-f794-4596-971f-4204f3fb11bb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.624982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.625996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.626478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.627012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.639968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:07.684110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efef576-4d27-4181-abe1-a78dc5f59588 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:07.708010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.710090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.723094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:07.723845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:07.724078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:07.724293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:07.725002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=b179c391ce574cd8836b43ea5bd94222) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:07.725316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:07.726748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:07.733657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 341/686] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/role_assignments?user.id=b179c391ce574cd8836b43ea5bd94222&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:07.764610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:07.791611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.837855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 346/687] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:07.850868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:07.969887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60cafd-3c2f-42a3-a20a-ef5011bfc683 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.001552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:08.015654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6eb0c-6931-4794-839e-477a1c99954e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.016987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.017797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.018084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.018355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.031911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.033362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:08.056048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:08.057080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:08.057974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:08.057974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:08.057974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:08.057974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 347/688] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:08.060828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:08.062949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:08.073595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.075244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.075745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.076226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.092908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.094987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:08.115387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:08.116598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:08.117721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:08.117721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:08.117721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:08.120302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 342/689] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:08.125858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:08.200620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b8c5-581b-4852-b7bf-3c2100c8cbfd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.242715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.243715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.244213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.244686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.245543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:08.246054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.247804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.263885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 348/690] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:08.270490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:08.319809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4020ea-b127-425d-8565-73518f470490 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.324416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:08.397597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.398137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.398421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.398697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.399386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:08.399694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.401068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.433655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 343/691] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:08.438920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:08.465309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb53aec-02ae-4647-a1c7-bbd23325d334 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.514258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.514921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.515207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.515487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.532270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:08.534714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.540210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:08.563771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:08.564841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:08.565726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:08.568756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 349/692] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:08.575203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:08.634049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0416-91b5-4399-9ac6-a1b9320eeb75 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.681318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.681852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.682132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.682405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.683247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.684686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.717434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 344/693] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:46:08 2022] POST /identity/v3/users => generated 254 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:08.771235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:08.780863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358e674-6d71-41eb-a948-ee380a823b84 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:08.832916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:08.833601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:08.833887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:08.834164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:08.834903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:08.835220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:08.836687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96efa8f-7485-4102-b4a0-fe6fe202ef5a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:08.853772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 350/694] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:08.860010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:08.994317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:09.003881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22701b-ce2b-4b96-be74-078f31578a33 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:09.040888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:09.041573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.041854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.042122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.042869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=d2ba853553814d49bf3c3a92c29fd27f) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:09.043181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:09.044603 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1697d4b-e0d7-4bf0-bb83-ac3cc01ae83c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:09.052561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 345/695] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:08 2022] GET /identity/v3/role_assignments?user.id=d2ba853553814d49bf3c3a92c29fd27f&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:09.531351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df5565-aa73-48e9-aba4-646d70d5a784 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.531900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df5565-aa73-48e9-aba4-646d70d5a784 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.531900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df5565-aa73-48e9-aba4-646d70d5a784 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.533261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 351/696] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:09.539190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.539652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.539763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.562593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:09.563972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:09.660573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 346/697] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:10.033032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df5565-aa73-48e9-aba4-646d70d5a784 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.179995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df5565-aa73-48e9-aba4-646d70d5a784 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:10.231146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b781575-32fc-4eeb-b89d-85b3ad753262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:10.231835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b781575-32fc-4eeb-b89d-85b3ad753262 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.232067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b781575-32fc-4eeb-b89d-85b3ad753262 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.232347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b781575-32fc-4eeb-b89d-85b3ad753262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.251709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b781575-32fc-4eeb-b89d-85b3ad753262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0b781575-32fc-4eeb-b89d-85b3ad753262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:10.256977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:10.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:10.258913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 352/698] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:10.267935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.305246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff7fac-7887-4451-ac1d-93d55eee510b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.305590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff7fac-7887-4451-ac1d-93d55eee510b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.305812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff7fac-7887-4451-ac1d-93d55eee510b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.307458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 353/699] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:10.312589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.312880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.313162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.332914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:10.334234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:10.354515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:10.400674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:10.420228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1af0e-45e3-4925-b3c0-b2a805946940 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:10.479456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 354/700] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:10.489662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:10.490227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.490511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.490787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.491493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:10.491799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:10.493167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4cce8-38d1-4e6d-b3af-fd3f6b4650cb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:10.510354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 347/701] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:10.515946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.666385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e449205-3e5b-40bd-b98c-fbd08cd1caa4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:10.715763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:10.716410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.716694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.716973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.733227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:10.734697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:10.735801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7bb88-2158-4ea0-b51c-597e3b4392c3 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.736263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7bb88-2158-4ea0-b51c-597e3b4392c3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.736344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7bb88-2158-4ea0-b51c-597e3b4392c3 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.739838 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 348/702] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:10.745502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:10.754873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.754873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.754873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:10.755214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:10.756073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:10.757002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:10.757002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:10.757002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:10.762864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 355/703] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:10.762864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.790397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:10.792110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:10.923076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 349/704] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:10.924373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.933038 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882fdc1-0daa-4011-9ab1-80941e835906 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.018109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.028139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.028744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.029031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.029308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.029958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:11.030268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.031760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.041863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeec5a-0d65-4082-9f7b-6b550ae5ad08 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.061042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 356/705] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:11.066380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.079013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.079936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.080521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.080995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.081783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.084452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:11.095978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:11.096987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:11.098599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:11.101855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 350/706] 10.222.0.72 () {56 vars in 1129 bytes} [Mon Sep 26 17:46:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:11.107914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.218240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1deb0d-d837-43af-afce-c3237a117afa None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.224144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.258960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.259954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.260435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.261419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.262277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:11.262788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.264395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.270686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 351/707] 10.222.0.72 () {56 vars in 1124 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:11.277805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.295221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.298782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7a08-1d5c-4871-9de5-77e403633dcc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.337035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.337727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.338015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.338292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.345937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.347462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:11.360665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:11.361646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:11.362647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 357/708] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:11.364437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.406531 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fce5b-17dd-4c3a-afd0-25434279d1ab None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.454922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.455958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.456462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.457699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] PATH_INFO: `/v3/services/network` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.458504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.460136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:11.471371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:11.472413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:11.473447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:11.481664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 352/709] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:11.481664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.495812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450a5d5-bd9e-4778-b2ff-90cd842959fd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.519237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.544585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.545437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.545870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.546299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.547626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:11.548085 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.549720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.556399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 358/710] 10.222.0.72 () {56 vars in 1124 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:11.557506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.576800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.616191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5aa7c-f60b-4ee9-84ab-dd23d67e5625 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.660959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.661846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.662383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.662797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.663812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:11.664380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.666419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237236e-09e1-48c1-b26d-194a759a8aaa None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.677459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a7205-134c-4f94-bd44-4bd12b435183 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.703319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 353/711] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:11.705014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa38e86-3bb7-4ff6-9447-e522a2974bac None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.705408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa38e86-3bb7-4ff6-9447-e522a2974bac None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.705788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa38e86-3bb7-4ff6-9447-e522a2974bac None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.709972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 354/712] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:11.711271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa38e86-3bb7-4ff6-9447-e522a2974bac None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.724601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.725240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.725595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.725873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.726594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] RBAC: Adding query filter params (name=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:11.726942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.729148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d5c26-5168-474c-b575-bd5df1685462 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:11.736981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 359/713] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:11.738329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.738623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.738927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.764397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:11.765327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:11.819215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.852092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 360/714] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:11.852798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:11.855595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa38e86-3bb7-4ff6-9447-e522a2974bac None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.894229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:11.904838 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:11.905764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:11.906286 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:11.906770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:11.959728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dfd51-68c7-44fc-9617-56408ab41f2e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:11.989181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:11.991459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.008588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.009640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.010129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.010601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.011488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] RBAC: Adding query filter params (type=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:12.011995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.013586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633b17-85bf-4352-b3da-46f0990661ef None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.018924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 361/715] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:12.020416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083e28b-9f26-41cc-a541-cbd9cfc2a237 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.020916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083e28b-9f26-41cc-a541-cbd9cfc2a237 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.021393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083e28b-9f26-41cc-a541-cbd9cfc2a237 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.027425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 362/716] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:12.029194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.029791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.030272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.049911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:12.052303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:12.091167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.109148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 355/717] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:12.115660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.130736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 363/718] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:12.136068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.157080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.255673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3d5a-de8c-43e5-9a21-0b6399462a77 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.257234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4bf4-f492-4ac5-b4ea-3b2b57e03e65 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.297569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.298256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.298545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.298813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.312240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.313806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:12.333717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:12.334770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:12.335745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:12.335745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:12.335745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:12.335745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 356/719] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:12.344101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.348787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.349480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.350780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.350780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.368059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:12.375017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:12.375933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:12.376761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:12.379870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 364/720] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:12.385614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.428068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.477638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c4f29-9cca-4ae8-ab1a-0b32c0bb8c1d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.515812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.516432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.516716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.516987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.517685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:12.517995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.520014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.525799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d352cbe-d7c9-4cbc-94d6-8f8375018e10 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.533232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 357/721] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:12.539396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.555242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.569546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.570655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.571493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.571971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.573115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:12.573641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.575741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.587818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 365/722] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:12.588966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.663818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794814-4f4e-4a84-8f08-02bf1e747132 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.704371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.712004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.712943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.713418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.713894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.716018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d013c-c7e3-458e-b28b-fcda5016fad4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.728440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:12.733964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:12.734983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:12.735836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:12.742770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 358/723] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:12.742770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.745675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.767533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.768171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.768458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.768981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.776646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.778195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:12.793481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:12.794471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:12.795395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 366/724] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:12.795961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.879264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6dc5-a53c-48db-843c-9ca529c37334 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.926376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.927164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.927453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.927732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.929017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5047d-73cd-4b4f-9b90-b75d322e8f78 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.940406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.943142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:12.965053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:12.966072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:12.967411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:12.967411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:12.967411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:12.973098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 359/725] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:12.974200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.982868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.982868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.982868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.982868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.983800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:12.983800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.983800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.994102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 367/726] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:12.995503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.015747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.057938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.110722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0c85e-894b-4f9a-9c22-bbe7da652505 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.153913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.154537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.154819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.155144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.155982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:13.156296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.157201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a839-0fed-43c6-b89a-3365a0c79ea9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.157773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.173348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 360/727] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:13.174356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.223706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.225195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.225727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.226151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.227216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:13.227672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.229981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.246824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 368/728] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:13.247839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.293744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c2d39-e004-4122-b6f5-b79a98897c3f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.313529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.338481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.339186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.339563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.340083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.340315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.353579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.355278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.371331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212dbd7-dfbc-4796-b1f4-e12e02721be2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:13.371978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:13.373099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:13.374203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:13.374203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:13.374203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:13.375837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 361/729] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:13.377238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.419683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.420547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.421003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.421424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.422823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1adfb9b41a1d4d31bccf5cc87b955526) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:13.423311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.425369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.434111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 369/730] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/role_assignments?user.id=1adfb9b41a1d4d31bccf5cc87b955526&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:13.435506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.518974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359eaa39-b9dd-4cac-b6f9-244759936ad0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.569422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b453a8-9267-41d0-aa89-c99cdcb4961b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.576004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.576926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.577372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.578464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.590910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.593261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.608260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.608875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.609156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.609428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.610079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:13.610334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:13.611420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:13.612471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:13.613005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.613005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.615730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 362/731] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:13.624688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.635005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.644391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 370/732] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:13.650269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.691039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.801213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c11af-c317-4489-a84d-ea2a9edde089 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.848717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b639f-a9d3-4b49-9b54-d69fecb9a27c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.859103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.859985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.860264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.861340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.862083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:13.862389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.864369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.884747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 363/733] 10.222.0.72 () {56 vars in 1149 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:13.890047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:13.928899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.929537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.933033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.933314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.949419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.952291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:13.984811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:13.985770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:13.986698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:13.989300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 371/734] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:13.996309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:14.055157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.088189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bf0ae-607d-4953-be05-d9d50f1e91ef None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.118849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.138057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a8e3-31e5-490e-bada-981a6189c9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.138785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a8e3-31e5-490e-bada-981a6189c9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.139145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a8e3-31e5-490e-bada-981a6189c9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.139430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a8e3-31e5-490e-bada-981a6189c9c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.172415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214504a-66a6-4970-a97e-113811ec5681 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.213785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056a8e3-31e5-490e-bada-981a6189c9c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.215329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056a8e3-31e5-490e-bada-981a6189c9c8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:14.216139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.217115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.217540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.218703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.219600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:14.220070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.221640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3f996-6514-4595-bc3c-2559a7ef705b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:14.248362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 372/735] 10.222.0.72 () {56 vars in 1135 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:14.248362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd22e-9d93-4d11-9c15-e25d4a13fecc None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.248362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd22e-9d93-4d11-9c15-e25d4a13fecc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.248362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dd22e-9d93-4d11-9c15-e25d4a13fecc None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.250900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 373/736] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:14.251916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd22e-9d93-4d11-9c15-e25d4a13fecc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:14.319641 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.336069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 364/737] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:14.337764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.338175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.338457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.363034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:14.364818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:14.380191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dd22e-9d93-4d11-9c15-e25d4a13fecc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.398279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.419167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb6fe8-9be8-4e09-9895-1583ba75663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.420822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb6fe8-9be8-4e09-9895-1583ba75663e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.421480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb6fe8-9be8-4e09-9895-1583ba75663e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.422011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb6fe8-9be8-4e09-9895-1583ba75663e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.451264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 365/738] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:14.452440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:14.507339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb6fe8-9be8-4e09-9895-1583ba75663e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.510106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb6fe8-9be8-4e09-9895-1583ba75663e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:14.617336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 374/739] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:46:14.623335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70f023-5fe7-433e-a8d0-71107fafa148 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.623922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70f023-5fe7-433e-a8d0-71107fafa148 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.624398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70f023-5fe7-433e-a8d0-71107fafa148 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.625204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd21d5-3190-4eae-8083-d03999edcd42 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.626353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 375/740] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:14.627279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70f023-5fe7-433e-a8d0-71107fafa148 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:14.646590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.648778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.670484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.671330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.671718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.672100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.684000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.685680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a29448f7-cb5e-48b7-8221-0e0e72cb9fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:14.702349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:14.703473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:14.704566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:14.704566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:14.704566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:14.706133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 366/741] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:14.708187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.708584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.709069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.728633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:14.730097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:14.742616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70f023-5fe7-433e-a8d0-71107fafa148 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.778180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.779081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.779525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.779935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.790531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.792023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:14.811335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 367/742] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:46:14.811335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:14.811608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:14.812497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:14.813398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:14.813398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:14.813398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:14.813398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 376/743] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:14.813398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:14.895852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.935491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d431-fed4-4a27-8c74-c958e26e9450 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.944120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d04c1-f89f-4f21-9f60-9ba206797f43 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:14.954145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:14.991132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.991977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.992407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.992827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.993011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:14.993624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:14.993900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:14.994094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.994201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:14.994862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:14.995186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:14.996025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:14.996584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.010462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 368/744] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:15.011341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.017222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 377/745] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:14 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:15.018224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.143219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1e6a-679e-4163-9a8a-8f720b289d43 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.167184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e01308-8e38-4d24-9802-8b2eaf20e993 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.192418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.193230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.193498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.193785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.194961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:15.195275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.197052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.198303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.213536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.214452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.214931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.215420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.216359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 369/746] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:15.217214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.224443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.226192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.238314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:15.238879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:15.239900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:15.241002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 378/747] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:15.241917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.340666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a4477-3129-4a94-8e11-db1cdfcc4691 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.377727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3263a-37b5-4b10-abb2-99cd99e6e15f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.377932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.378438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.380053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.380507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.381204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.382586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.405399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 370/748] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:15 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:15.406760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.428690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.429235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.429492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.429819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.437840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.439458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:15.453047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:15.454245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:15.455351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 379/749] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:15.464005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.568503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.573526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.652786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5a7e8-c9ad-4bcb-a417-cc15f2804262 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.656210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcfd89-a32b-49e9-82af-26331c89359b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.688443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.689108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.689396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.689772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.690477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:15.690787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.693176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.699596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.700346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.700728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.701107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.702392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:15.702828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.704829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.707928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 371/750] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:15.713621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.719892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 380/751] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:15.726301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.851552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.855191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204f6ed-a6a2-4cd4-ad93-313d7a3ce3d7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.867316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbaec41-724f-4b42-80f3-7f72de76b719 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.896411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.897155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.897447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.897855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.898584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=d2ba853553814d49bf3c3a92c29fd27f) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:15.898935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.900633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb9a2e-033e-4198-b513-8b8b64a35dba None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.907468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 381/752] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/role_assignments?user.id=d2ba853553814d49bf3c3a92c29fd27f&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:15.937017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.937717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.938002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.938278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.939157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1de6964e61bd46eca46e1db919ae14c3) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:15.939412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.940976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bf0ce-d2b2-415a-8a52-9fbca4fe3f21 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.945755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.949885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 372/753] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/role_assignments?user.id=1de6964e61bd46eca46e1db919ae14c3&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:16.164602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abb5c-7057-44a8-983a-fa846caf229e None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:16.164912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abb5c-7057-44a8-983a-fa846caf229e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:16.165384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80abb5c-7057-44a8-983a-fa846caf229e None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:16.167445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 382/754] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:16.173700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:16.174014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:16.174250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:16.195679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:16.196813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:16.268798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 373/755] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:16.600777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80abb5c-7057-44a8-983a-fa846caf229e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:16.715629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80abb5c-7057-44a8-983a-fa846caf229e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:16.752724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:16.753358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:16.754189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:16.754528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:16.765272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:16.770795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:16.771730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:16.772607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 383/756] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:16.779750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:16.888857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f982115-1d69-483a-8a3b-165a149bca9d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:16.922344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:16.925291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:16.926664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:16.927072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:16.927346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:16.928018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:16.928322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:16.929909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:16.938742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 374/757] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:16.948167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:16.990555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:17.100303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e21b-b1b2-43b3-b764-8e9b9aa81fdf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:17.146400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:17.146976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:17.147256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:17.147567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:17.163442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:17.165800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:17.192750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:17.193734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:17.194914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:17.194914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:17.194914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:17.197660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 384/758] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:17.202756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:17.355424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc77501-ced3-4e2a-84d6-0f0db5ff7db2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:17.408035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:17.408988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:17.409493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:17.410516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:17.411623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:17.412156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:17.413719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a5ed0-250c-4582-a85c-4a3314f4e65d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:17.430820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 375/759] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:17.436760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:17.548097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:17.579299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bda3c0-1cc7-488e-9d40-4f4bb8c56167 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:17.621034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:17.621727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:17.622009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:17.622280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:17.634062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:17.635636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:17.644148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d366d4-88b4-4dcf-8f65-334814fd01ba None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:17.645254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d366d4-88b4-4dcf-8f65-334814fd01ba None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:17.645732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d366d4-88b4-4dcf-8f65-334814fd01ba None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:17.649594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 376/760] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:17.656198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:17.657155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:17.658781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 385/761] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:46:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:17.662593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:17.662903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:17.663179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:17.666326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:17.689363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:17.692793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:17.793578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:17.812051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ecde7-c69d-4e7b-925b-9ad26c9389de None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:17.836565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 377/762] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:17.856887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:17.857893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:17.858394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:17.858910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:17.859838 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:17.860367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:17.862083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:17.865052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:17.889021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 386/763] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:46:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:17.893486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.067558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6636a-f58c-4aa1-a216-842f3247f2fa None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.115080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.115725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.116007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.116283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.126810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.128273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:18.150193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:18.151251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:18.152141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:18.152141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:18.152141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:18.155584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 378/764] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:18.163092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.165134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.216683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:18.304035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531781c-ce13-46d7-a0a5-ae8cd1db9c2c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.304339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb1bcc-b0ea-4ec2-8976-b1b362340bf9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.350435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.351202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.351487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.351760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.352479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:18.352960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.355008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.371584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 387/765] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:18.376534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.388156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.388803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.389084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.389401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.389995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.391497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:18.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:18.411798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:18.412626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:18.416845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 379/766] 10.222.0.72 () {56 vars in 1129 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:18.436228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.487815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:18.523732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e11f4-1ecc-4a87-a354-a2dbf86bbe75 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.569245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.569734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.570018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.570328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.579401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.581011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:18.595263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:18.596227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:18.597121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 388/767] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:18.603004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.705514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:18.712472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d671c2-f309-444a-bb41-1c8a4cdfee87 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.721359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb256a-e90e-45dc-87f6-2a9c31440c15 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.737255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:18.751031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.751659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.751943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.752219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.752862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:18.753173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.754614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d938-44a0-4cd8-849d-fd9fd71ee7fe None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.759025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.771032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 380/768] 10.222.0.72 () {56 vars in 1124 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:18.771210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.771210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.771210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.771454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c9355-7c3f-44c8-9c9e-b39796487632 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.771540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:18.771636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c9355-7c3f-44c8-9c9e-b39796487632 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.771636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c9355-7c3f-44c8-9c9e-b39796487632 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.771804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.771804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b3f6b-1e0a-4760-8a3b-766eb7da2d07 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.771956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 381/769] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:18.771956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407bb09-6e1d-4f27-a51c-0277f783b6db None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.771956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407bb09-6e1d-4f27-a51c-0277f783b6db None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.771956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407bb09-6e1d-4f27-a51c-0277f783b6db None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.771956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 382/770] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:18.771956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407bb09-6e1d-4f27-a51c-0277f783b6db None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.790636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 389/771] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:18.790636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d455b2d-b9d5-439d-90ff-3217d660cbca None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.790636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d455b2d-b9d5-439d-90ff-3217d660cbca None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.790636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d455b2d-b9d5-439d-90ff-3217d660cbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.807466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8d455b2d-b9d5-439d-90ff-3217d660cbca None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:18.808681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8d455b2d-b9d5-439d-90ff-3217d660cbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:18.875054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407bb09-6e1d-4f27-a51c-0277f783b6db None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.875224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 390/772] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:18.876363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f70b-7e53-4d9e-b562-6549ab042d4d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.876643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f70b-7e53-4d9e-b562-6549ab042d4d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.876911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f70b-7e53-4d9e-b562-6549ab042d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.895118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-06b2f70b-7e53-4d9e-b562-6549ab042d4d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:18.896315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-06b2f70b-7e53-4d9e-b562-6549ab042d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:18.925773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.926327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.926611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.926905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.927530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:18.927830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.929177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.940025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 383/773] 10.222.0.72 () {56 vars in 1124 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:18.940025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.970919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:18.973211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 391/774] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:18.974482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e89c-33db-4215-b17e-8a939cb455ee None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.974773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e89c-33db-4215-b17e-8a939cb455ee None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.975076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e89c-33db-4215-b17e-8a939cb455ee None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.978891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 392/775] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:18.978891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e89c-33db-4215-b17e-8a939cb455ee None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.029144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.062627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2d7ee-c9ad-41c4-8d20-f925d1d1c162 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.106263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848d9e-5d44-4a5a-a648-b5e28c1fffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.106951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848d9e-5d44-4a5a-a648-b5e28c1fffc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.107240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848d9e-5d44-4a5a-a648-b5e28c1fffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.107663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848d9e-5d44-4a5a-a648-b5e28c1fffc2 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1adfb9b41a1d4d31bccf5cc87b955526/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.109680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848d9e-5d44-4a5a-a648-b5e28c1fffc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=1adfb9b41a1d4d31bccf5cc87b955526, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.111380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848d9e-5d44-4a5a-a648-b5e28c1fffc2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.117361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e89c-33db-4215-b17e-8a939cb455ee None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.135183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 384/776] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:18 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1adfb9b41a1d4d31bccf5cc87b955526/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:19.136995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.136995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.136995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.168092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:19.169307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:19.182868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.182868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.182868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.182868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.183630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] RBAC: Adding query filter params (interface=public, service_id=a9a8a074be7341eb8a2dea5cc608b03c, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:19.183630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.183630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.198946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 393/777] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/endpoints?service_id=a9a8a074be7341eb8a2dea5cc608b03c&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:19.200421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.271509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 385/778] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:19.271509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.291858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.358862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.362008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d3295-1f54-4daa-abb2-8409878733c9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.395521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad8594-230c-4e99-a3f5-5224225aa996 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.404977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.406350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.406892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.407812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] PATH_INFO: `/v3/services/network` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.408915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.411592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:19.428777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:19.429800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:19.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:19.436853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 394/779] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:19.436853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.440054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.440760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.441268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.442180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.442954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.444752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.451168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 386/780] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:19.452117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.557324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.574568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31229d9e-c9f9-4bd3-84a1-632740dac1ec None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.585566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbe9f1-0827-4c45-8b3e-6cd1cb85e6ac None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.629618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.630339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.630622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.630934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.635470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.636442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.636981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.637723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.638603 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:19.639139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.640754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.642813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:19.647809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 387/781] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:19.648509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:19.649424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:19.650295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:19.652118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.654460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 395/782] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:19.655491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.689092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.790201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f0229-9eec-4fd7-b80d-6a2f01e032e1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.802353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ed7e4-158d-40bf-b8ed-6eaf724386b4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.834721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.835796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.836329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.836800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.850416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:19.855778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:19.856784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:19.857608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:19.860495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 388/783] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:19.861686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.867347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.957804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.958324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.958603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.960817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.960817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] RBAC: Adding query filter params (type=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:19.960989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.962308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.984056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 396/784] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:19.984906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.986965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c4e2-be9a-4efc-872d-2139f0f220b2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.033080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.034096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.035700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.036195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.037534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:20.038060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.039987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.054125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 389/785] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:20.055605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.115218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:20.119831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dde74b-4961-4a7e-9cf5-b6153f534925 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.139536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:20.161359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.162055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.162294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.162571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.163334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:20.163643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.165355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.176242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 397/786] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:20.177325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.190305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c3d36-a4c9-40d4-8eda-50226f7f36d6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.228852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.229393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.229677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.229954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.230659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] RBAC: Adding query filter params (interface=public, service_id=108f173971c747f2ad10a1197bea7f13, region_id=RegionOne) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:20.230997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.232364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.243701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 390/787] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/endpoints?service_id=108f173971c747f2ad10a1197bea7f13&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:20.244535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.295211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e279648-5019-4d41-b367-dc1c4584ddbd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.334429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.335191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.335476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.335779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.347113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.349521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.355265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04acbd-f1fa-4cbc-a38b-0f3c468d3759 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.358697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:20.368097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:20.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:20.369970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:20.369970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:20.369970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:20.369970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 398/788] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:20.370670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.399407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:20.407365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.408006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.408538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.408819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.421954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.423566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:20.444666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:20.446933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:20.449386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:20.449386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:20.449386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:20.449386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 391/789] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:20.449386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.497769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f964-a52d-408e-b498-4ed92934dba5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.541438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.542958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.543248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.543526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.544136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.545507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.554928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 399/790] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:20.555822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.607308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcba292-3598-45cc-ab82-8d66674aafa5 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.671747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.675297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.676222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.676611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.677382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:20.677687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.679452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.697876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:20.716441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 392/791] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:20.718491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.779736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b35ab-7831-4a0b-8eb8-d0d1ad4984c6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.795624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:20.831549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.832846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.833127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.833399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.834172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:20.834478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.837211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.854966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 400/792] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:20.859520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.879029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcafb5-b300-44fb-9ca6-b128b6317d92 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:20.922538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:20.923609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:20.924350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:20.924839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:20.935133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:20.937119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:20.953355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:20.954329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:20.955306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:20.958432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 393/793] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:20.966161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:20.991653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d0607-eb60-4952-a1e4-5e42b6914648 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.039667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.061296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.062494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.063787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.064443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.079970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.083313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:21.098960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:21.099979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:21.100977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:21.103602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 401/794] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:21.109412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:21.172992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a1b3-2492-4b9e-8aed-40aee5e6a3b2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.174618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.213753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.214366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.214646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.214943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.215724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:21.215973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.217616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:21.236796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 394/795] 10.222.0.72 () {56 vars in 1149 bytes} [Mon Sep 26 17:46:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:21.243059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:21.275194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184923c-a43f-45aa-be84-f841868dd326 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.339203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.340633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.341122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.341651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.342596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:21.343143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.344776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:21.366228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 402/796] 10.222.0.72 () {56 vars in 1135 bytes} [Mon Sep 26 17:46:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:21.373057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:21.430307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd07cc-0a58-4942-a517-c9106d1a62cf None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.478805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.488593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.491290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.491573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.491899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.510521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.512154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-fbbe5946-dcd3-4004-9a6c-4f2ef48a04f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:21.544190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:21.545209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:21.546124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:21.546124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:21.546124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:21.547022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 395/797] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:21.549934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e9b2-341f-442e-b3f0-60708a2c3773 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.550218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e9b2-341f-442e-b3f0-60708a2c3773 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.551398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e9b2-341f-442e-b3f0-60708a2c3773 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.559835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 396/798] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:21.561013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e9b2-341f-442e-b3f0-60708a2c3773 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:21.579199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.594292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c04ea8-6a2f-406e-8b13-b06ce7415062 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.636074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.636970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.637249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.637509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.650493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.652296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-7307fa33-3db0-4028-9d89-bbc4509bec43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:21.672861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:21.675137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 403/799] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.677786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.700114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e9b2-341f-442e-b3f0-60708a2c3773 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.706048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:21.707705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:21.742936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.743931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.744526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.745132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.745856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:21.746234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.747807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:21.761916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 397/800] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:21.762931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:21.795955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 404/801] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:21.798014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:21.839243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.849745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.898379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95a6fd-28e8-4327-8b28-e91cfa20baa8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:21.973183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:21.973821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.974103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.974377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.975849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:21.975849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:21.980207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69b8b3-a9ee-4f18-b3c9-2dd56828313e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.000990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 398/802] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.005876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56cd2e-d4e0-4363-a9df-b8ec97416c66 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.006101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56cd2e-d4e0-4363-a9df-b8ec97416c66 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.006371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56cd2e-d4e0-4363-a9df-b8ec97416c66 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.007938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 399/803] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:22.008690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56cd2e-d4e0-4363-a9df-b8ec97416c66 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.013748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407f21e-ac42-4d2f-a75d-0b3f2747cf05 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.060300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.061570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.062059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.062892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.072529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.075278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d6e1926c-0e74-4b9d-921b-27c5d04c0b3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:22.091425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:22.092347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:22.093250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:22.096105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 405/804] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:22.098253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.098756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.099254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.122128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:22.124329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:22.129572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56cd2e-d4e0-4363-a9df-b8ec97416c66 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.160899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.166287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.167609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.167609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.167609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.174381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.175927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:22.187722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:22.188622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:22.189527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 400/805] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:22.190016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.203990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 406/806] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:22.204595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.222055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.313337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64523-1e33-4c90-9891-893924e24de8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.338424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfe87b-07a9-414e-ba91-9fb75f62fa20 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.355346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.356500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.357380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.357890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.358821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:22.359357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.361815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.376817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.377427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.377654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.377926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.380012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 401/807] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.381043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.395558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:22.400785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:22.401718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:22.402588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:22.406801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 407/808] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:22.407612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.453401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.501193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.514562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4ad8b-8402-46c3-bd69-ab7adae7633d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.528805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae3b9-bb2b-4099-9a1e-c5ffef669124 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.552411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.553036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.553586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.553867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.554568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:22.554913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.556353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.566968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.567627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.567971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.568253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1de6964e61bd46eca46e1db919ae14c3/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.574771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 402/809] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.575796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.588880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=1de6964e61bd46eca46e1db919ae14c3, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.590371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.604447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 408/810] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:22 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1de6964e61bd46eca46e1db919ae14c3/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:22.611831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.717797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0786f6-2c03-42a7-bb25-c60bdc86f4d5 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.737045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.753090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b4c0c-00e9-4579-adcc-cc419b8a44a5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.771299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.772499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.773547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.774250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.776405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:22.777149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.780294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.786557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.796002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 403/811] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.797183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.811985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.812245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.812493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.812772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.813361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.814740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:22.825062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:22.826333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:22.827417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 409/812] 10.222.0.72 () {56 vars in 1129 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:22.828407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.936777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0615a-2773-4fdc-936e-5e3148cb57a3 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.940646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d614a4-8edd-4820-87bf-bfa5632ac676 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.987510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.989069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.989569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.990028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/d2ba853553814d49bf3c3a92c29fd27f/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.991577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.992378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.992592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=d2ba853553814d49bf3c3a92c29fd27f, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.992781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.993125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.994710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.006765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.008489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.013558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 404/813] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:22 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/d2ba853553814d49bf3c3a92c29fd27f/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:23.014819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:23.040238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:23.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:23.042619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:23.042619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:23.042619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:23.043364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 410/814] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:23.048733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.062748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:23.123748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:23.161517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9165d-a5d4-47ac-bb9e-fd53e100da4e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:23.200468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d5f72-5241-4e27-ae57-fbdab9388f96 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:23.214670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:23.215340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.215630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.215910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.216858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:23.217167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.219545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021b934-9989-41d1-aa00-e7e9a0f7fffa None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.227179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 405/815] 10.222.0.72 () {56 vars in 1124 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:23.228754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3370e9-ef4a-47d0-846d-e68c5d84b7b8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.229038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3370e9-ef4a-47d0-846d-e68c5d84b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.229311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3370e9-ef4a-47d0-846d-e68c5d84b7b8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.234945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 406/816] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:23.234945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3370e9-ef4a-47d0-846d-e68c5d84b7b8 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.250714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:23.251362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.251647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.251924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.252668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:23.252981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.255159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fb426-909f-4949-8e91-238043e2f3aa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.268948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 411/817] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:23.271406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.271898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.272387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.290810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:23.292040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:23.360095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:23.367123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3370e9-ef4a-47d0-846d-e68c5d84b7b8 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:23.375008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 412/818] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:23.376039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.405968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:23.406656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.406969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.407245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.407884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:23.408188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.409894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.415781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 407/819] 10.222.0.72 () {56 vars in 1124 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:23.421194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.424990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:23.540759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078c94-49c9-438d-8bfe-f00fe80e73ee None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:23.550306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1c03e-a397-48ec-a27e-3f2877279edb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:23.587403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:23.588045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.588325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.588602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.589299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.591391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.622868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 408/820] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:23.623911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:23.624616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.624793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.625276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.634926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.636515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:23.651177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:23.653548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:23.654740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 413/821] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:23.661170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.704082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:23.820103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.842253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b2dc8-6c2f-435a-b90e-75672deff73c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:23.882768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:23.883420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:23.883702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:23.883975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:23.885062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:23.885062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:23.886440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:23.906160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 409/822] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:23.912354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:23.951867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:23.964613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4c975-88ed-4730-9176-26b8e1bb6cca None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.004427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:24.014137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.014777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.015089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.015398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] PATH_INFO: `/v3/services/network` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.016445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.018100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:24.029654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:24.030660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:24.031537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 414/823] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:24.037957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:24.046678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c2907-5512-4141-8fb8-8335e0271f48 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.088479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.089551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.090076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.090612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.165364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.167184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.217674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88da39f-5df0-4f51-8f02-bcfa0aa97cbc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.286174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.286790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.287096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.287371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.288004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] RBAC: Adding query filter params (name=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:24.288308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.289751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.295259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:24.309109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 415/824] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:24.314561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:24.333085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 410/825] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:24.338877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:24.363967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:24.448093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a280-64c7-4c37-85d2-66ba940b943f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.485899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.486259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b98d0-28ef-40a0-b5ac-e3c54c19d712 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.487492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.488621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.489134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.500788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.502737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:24.520466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:24.521470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:24.522388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:24.522388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:24.522388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:24.523212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.523825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.524114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.524392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.525021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] RBAC: Adding query filter params (type=network) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:24.525387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.525817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 411/826] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:24.530250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.531420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:24.532190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 416/827] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:24.538050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:24.561001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:24.630977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:24.652860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e028b-e2e7-4568-9e5a-1d196bd56bbe None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.719043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.719944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.720369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.720785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.721597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.723665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.765487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 417/828] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:24.803508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93561129-1d51-43be-8d69-aec9c198c753 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:24.850651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:24.851666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:24.852142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:24.852632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:24.853489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:24.853995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:24.855626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:24.869045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 412/829] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:24.875009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.019009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb4478-d261-45cd-8660-77940b66111f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.055314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.055953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.056284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.056562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.063613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:25.065062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:25.066921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ab02c475-ddf9-49b1-942c-57b0b606d6eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:25.078922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:25.079972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:25.080859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 418/830] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:25.093516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.252024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:25.287321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c27aa-d204-45fb-9771-6802d65b7f46 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.340818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77424184-fa15-4aab-981f-a25ba03f16d9 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.341262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77424184-fa15-4aab-981f-a25ba03f16d9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.341661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77424184-fa15-4aab-981f-a25ba03f16d9 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.343865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 419/831] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:25.349226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e6e92-9da8-48ab-a941-725f57840ea6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.349655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e6e92-9da8-48ab-a941-725f57840ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.349767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e6e92-9da8-48ab-a941-725f57840ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.352044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.352945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.353457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.353734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.354429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:25.354738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:25.356189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:25.372959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ef4e6e92-9da8-48ab-a941-725f57840ea6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:25.374171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ef4e6e92-9da8-48ab-a941-725f57840ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:25.376125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 413/832] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:25.382818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.448330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 420/833] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:25.496956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa999e7-933f-4b43-b856-e7a76eddb724 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.536005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.536537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.536815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.537087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.537874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1adfb9b41a1d4d31bccf5cc87b955526) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:25.538185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:25.539864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf318740-2bbf-4373-af7b-3f945a9e0d1f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:25.550097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 414/834] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/role_assignments?user.id=1adfb9b41a1d4d31bccf5cc87b955526&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:25.612350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0596b70-b31a-4332-8b46-32771eb51653 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.612909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0596b70-b31a-4332-8b46-32771eb51653 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.613019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0596b70-b31a-4332-8b46-32771eb51653 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.615177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 421/835] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:25.621107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.621649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.621649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.629447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:25.641669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:25.642917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:25.718449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 415/836] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:25.760403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0596b70-b31a-4332-8b46-32771eb51653 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.869901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0596b70-b31a-4332-8b46-32771eb51653 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.899707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:25.909245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.910782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.911439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.911948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.923109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:25.929192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:25.930158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:46:25.931096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:25.937868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 422/837] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:25.939392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.008761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.054705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ffe72-6a1e-4216-b039-d91671fd4922 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.095262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.096253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.096740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.097257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.098658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:26.099194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.100783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.102526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.111017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 416/838] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:26.117498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.136463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b862-3dc4-46d8-be02-e909f8d3e975 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.180286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.181405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.181834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.182247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.198623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:26.205197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:26.206133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:26.207629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:26.208392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 423/839] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:26.214233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.242636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449c1f2-b12b-4ce8-91e6-95798a8d8aec None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.279163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.282959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.283808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.284258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.284749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.297393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.299189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:26.314664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:26.315786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:26.317227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:26.317227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:26.317227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:26.320009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 417/840] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:26.325478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.349964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30bdb8-1a39-40dc-a0cd-017fba1445fb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.384118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.401955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.402506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.403213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.403500 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.404429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:26.404754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.406136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.422925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 424/841] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:26.424041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.451082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25465f-7b46-45d4-9aa2-a2a50f04b008 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.488927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.489535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.489819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.490095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.490809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:26.491147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.492636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3349439-4a54-4b08-8dde-2ccd555888e6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.509564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 418/842] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:26.511447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62af98-9856-4577-994c-18c99408d56f None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.512488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62af98-9856-4577-994c-18c99408d56f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.512970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62af98-9856-4577-994c-18c99408d56f None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.514676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 419/843] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:26.515790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62af98-9856-4577-994c-18c99408d56f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.535292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d69f98-2da4-432e-a774-6a4dcc5bf298 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.546821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.573718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.574432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.574717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.575187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.588593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.590154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-cb4cbe64-aea5-4ae5-8614-176f3479395a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:26.606756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:26.607795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:26.608697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:26.608697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:26.608697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:26.608697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 425/844] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:26.609345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.609345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.609587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.627791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:26.629080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:26.633342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62af98-9856-4577-994c-18c99408d56f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.642814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.670250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.670881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.671188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.671465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.679248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.680685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:26.695645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:26.696595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 420/845] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:26.697537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.723914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 426/846] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:26.723914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.806349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a38d0-2574-47d8-80fa-500364dc98f0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.830917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.840609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43e52-b457-44f2-8fbf-0d84893f4eab None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.844712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.845523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.845966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.846399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.847260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:26.848287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.849916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.862862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 421/847] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:26.877176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.885228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.885887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.886176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.886511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.887429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:26.887746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.889737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.915981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 427/848] 10.222.0.72 () {56 vars in 1149 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:26.921121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.921403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.015050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8ea34-69b6-430f-a8ec-d4390dc4e46b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.041692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca912-253b-4780-a2c1-02d14608f86e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.055030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.055937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.056429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.056912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.064154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.066010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.076521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.077130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.077413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.077691 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:27.079216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:27.080234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:46:27.081349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:27.082238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 422/849] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:27.083672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.103247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.156725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.158089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.177131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.192486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd0aa8-5562-4acf-a5c7-ad379fd98903 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.231412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.232824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.233109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.233384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.233976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.235410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:27.245605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:27.246705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:27.247593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 423/850] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:27.248404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.270510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 428/851] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:27.271290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.325543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.372468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb20b8-03ed-49f9-9218-c413f389e31b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.380444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303976-c0d9-43ba-b5ca-b1febeba3128 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.408137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.408779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.409069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.409413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.410800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:27.411140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.412625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.417409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.418013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.418291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.418561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.419210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:27.419510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.420855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.430173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 429/852] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:27.430173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.435447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 424/853] 10.222.0.72 () {56 vars in 1135 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:27.436224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.448794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.538799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11036f0a-556a-44bf-96e1-323dd1e4dc59 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.540875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a4ebe-e39a-4b32-a5d2-60f5c21b97b7 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.555884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.576513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.577121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.577400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.577671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.578293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:27.578592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.579854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.579980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.580539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.580819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.581202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.584935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 425/854] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:27.585725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.611876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.614497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:27.636688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:27.638171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:27.639143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:27.639143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:27.639143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:27.639143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 430/855] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:27.645530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.689096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.720780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23ac21-dde1-47c4-907e-9effbe8919d1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.762312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716158fc-8eb2-42bc-a39a-46f34e7b66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.763098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716158fc-8eb2-42bc-a39a-46f34e7b66a5 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.763452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716158fc-8eb2-42bc-a39a-46f34e7b66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.763790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716158fc-8eb2-42bc-a39a-46f34e7b66a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.772726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e2247-a8fd-4ecd-8716-e05044095e2e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.817352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:27.818421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.819214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.819689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.820562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:27.821077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.822501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.836638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 431/856] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:27.842392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:27.855925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716158fc-8eb2-42bc-a39a-46f34e7b66a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:27.857375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716158fc-8eb2-42bc-a39a-46f34e7b66a5 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:27.883293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.941486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:27.958994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18fc67-ff02-46c2-8c5a-7a1811bd9965 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:27.973737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 426/857] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:27.975050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d556f-1f20-4794-ad6f-d1c838120bf9 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:27.975338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d556f-1f20-4794-ad6f-d1c838120bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:27.975613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d556f-1f20-4794-ad6f-d1c838120bf9 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:27.981061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 427/858] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:27.981061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d556f-1f20-4794-ad6f-d1c838120bf9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.007466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.008156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.008443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.008737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.015941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.017388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a5915df6-bce7-4bab-8163-55ee6562d8b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:28.030979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:28.031926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:28.032822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 432/859] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:28.033981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73f256-0663-4641-8515-7018a3544037 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.033981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73f256-0663-4641-8515-7018a3544037 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.033981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73f256-0663-4641-8515-7018a3544037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.053026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fa73f256-0663-4641-8515-7018a3544037 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:28.054175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fa73f256-0663-4641-8515-7018a3544037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:28.095332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d556f-1f20-4794-ad6f-d1c838120bf9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.126742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 433/860] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:28.126742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73f256-0663-4641-8515-7018a3544037 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.138619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.139499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.140416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.140843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.149118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:28.156316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.159313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:28.179984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:28.181201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:28.182344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:28.182344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:28.182344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:28.187823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 428/861] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:28.188757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.218544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:28.251475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73f256-0663-4641-8515-7018a3544037 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.294012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.294627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.294929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.295205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.295890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:28.296196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.297797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.317035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 434/862] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:28.318042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.321528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b25ba9-fe91-4959-bc99-50d81137b01e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.372149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.373635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.374400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.374936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.376073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:28.376964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.378525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.396202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 429/863] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:28.397726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.406621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:28.432404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd0ccb-37a1-435f-9afd-f8cdad7cbd9f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.470599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.471515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.471998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.472800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.473518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1de6964e61bd46eca46e1db919ae14c3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:28.473817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.475269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.481320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 435/864] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/role_assignments?user.id=1de6964e61bd46eca46e1db919ae14c3&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:28.481996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.502061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:28.526388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1016ab-ddce-4818-a46f-937d8113f0e9 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.574717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.575469 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.575748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.576021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.576930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.579487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:28.592297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:28.593384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:28.594300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:28.599194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 430/865] 10.222.0.72 () {56 vars in 1132 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:28.600647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.625329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6cd1a-9602-4eff-b9e3-26ab9d599515 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.678354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.681471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.681471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.681471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.689696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.689836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.693677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:28.706415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:28.707460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:28.708393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 436/866] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:28.714585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.740056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7aefc-ff1c-4f16-b9ed-63a697be5a81 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.783299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.783921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.784201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.784473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.785501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:28.785628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.787144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.794879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 431/867] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:28.801357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.824608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:28.849379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c07687-84de-4b62-9f8b-798457977efb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.894671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.896170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.896587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.897026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.898069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:28.898698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.900336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:28.919957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 437/868] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:28.921410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:28.936259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4152f-5f88-45a1-8873-cd9b212ab9e4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:28.991146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:28.991875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.992258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.992644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.993545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:28.993955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:28.995754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f63819-8c9b-4c46-a3d8-4fdd08fc5765 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:29.004385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 432/869] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:29.009561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:29.068725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5f0bc-58f3-487e-94a7-ba08a3ce7160 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:29.108717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:29.109335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:29.109616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:29.109889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:29.110618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=d2ba853553814d49bf3c3a92c29fd27f) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:29.110955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:29.112403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b3768-1812-4ff2-a83f-61d53b92d27b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:29.120300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 438/870] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:28 2022] GET /identity/v3/role_assignments?user.id=d2ba853553814d49bf3c3a92c29fd27f&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:30.003415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c7a56-5b29-468c-8e27-429ddda73a90 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:30.003952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c7a56-5b29-468c-8e27-429ddda73a90 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:30.004058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c7a56-5b29-468c-8e27-429ddda73a90 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:30.005593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 433/871] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:30.011562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db489b85-3651-4062-b232-410b7599b349 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:30.012053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db489b85-3651-4062-b232-410b7599b349 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:30.012533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db489b85-3651-4062-b232-410b7599b349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:30.033737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-db489b85-3651-4062-b232-410b7599b349 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:30.035548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-db489b85-3651-4062-b232-410b7599b349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:30.109447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 439/872] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:30.382557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c7a56-5b29-468c-8e27-429ddda73a90 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:30.467611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:30.492800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c7a56-5b29-468c-8e27-429ddda73a90 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:30.528386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2def797-2535-45da-8031-ff6f209044ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:30.529057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def797-2535-45da-8031-ff6f209044ee None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:30.529337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def797-2535-45da-8031-ff6f209044ee None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:30.529610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def797-2535-45da-8031-ff6f209044ee None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:30.530278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2def797-2535-45da-8031-ff6f209044ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:30.531723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2def797-2535-45da-8031-ff6f209044ee None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:30.549233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 434/873] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:30 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:30.932762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba27da-feb7-4c4f-be72-f082f7b32f86 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:30.933169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba27da-feb7-4c4f-be72-f082f7b32f86 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:30.933357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba27da-feb7-4c4f-be72-f082f7b32f86 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:30.934908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 440/874] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:30.940363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d7bb4-6314-4881-8d8b-731ddbc58712 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:30.940655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d7bb4-6314-4881-8d8b-731ddbc58712 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:30.940932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d7bb4-6314-4881-8d8b-731ddbc58712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:30.961694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f98d7bb4-6314-4881-8d8b-731ddbc58712 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:30.962916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f98d7bb4-6314-4881-8d8b-731ddbc58712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:31.045845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 435/875] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:31.321006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba27da-feb7-4c4f-be72-f082f7b32f86 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:31.382102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:31.426421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd432-6db6-420f-a58a-112b33e8d9fb None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.426749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd432-6db6-420f-a58a-112b33e8d9fb None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.427108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd432-6db6-420f-a58a-112b33e8d9fb None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.428686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 436/876] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:31.433361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba27da-feb7-4c4f-be72-f082f7b32f86 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:31.433628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11456431-9280-4b80-a1e0-aa9ffdb7dcac None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.433929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11456431-9280-4b80-a1e0-aa9ffdb7dcac None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.434204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11456431-9280-4b80-a1e0-aa9ffdb7dcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.456168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-11456431-9280-4b80-a1e0-aa9ffdb7dcac None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:31.457656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-11456431-9280-4b80-a1e0-aa9ffdb7dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:31.478684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:31.479767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.480268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.480759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.499222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:31.506403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:31.507586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:31.508626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:31.511090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 441/877] 10.222.0.72 () {56 vars in 1120 bytes} [Mon Sep 26 17:46:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:31.518891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:31.536084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:31.555040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 437/878] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:31.600478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34aeba-e385-4ceb-8e2b-e1374c7ba860 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.600651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34aeba-e385-4ceb-8e2b-e1374c7ba860 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.600888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34aeba-e385-4ceb-8e2b-e1374c7ba860 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.602504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 438/879] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:31.608127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.608546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.608920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.627958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:31.629792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:31.641687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2215d9f-04d2-4cfb-b69a-d82f28ed102c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:31.661270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:31.688353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:31.689762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.690358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.691530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.692444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:31.693183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:31.695823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:31.712095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 442/880] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:46:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:31.712295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 439/881] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:31.713713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:31.819216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f3506-f7dd-46d4-8d32-27199ae424af None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:31.826407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:31.846324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:31.858122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:31.858689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.859061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.859671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.870538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:31.872132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:31.889330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:31.890345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:31.895380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:31.895380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:31.895380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:31.895380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 440/882] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:31.897732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:31.966881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b39973-9044-42a0-8e73-74fe829ce08b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.008274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.008877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.009161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.009434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.010092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.011502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.021190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573aed5c-2e7c-4d64-b2a9-df3f4bf2ea0d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.029244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 443/883] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:31 2022] POST /identity/v3/services => generated 269 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:32.030061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.082285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.088589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.090940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.091319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.091622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.092296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:32.093069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.095058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.113203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 441/884] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:32.120103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.164570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.187589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5364-a2f9-4c5f-a8a5-7dc2f49e87cf None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.235325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.236320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.236844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.237185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.237912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.240689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:32.249708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:32.250756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:32.251658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 444/885] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:32.257415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.262335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4836b47-5fa5-4219-aa48-3d924eb8c845 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.305394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.306935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.307501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.307781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.315583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.317923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:32.332585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:32.333579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:32.334504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 442/886] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:32.339818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.400529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.405035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcda268-eb9a-4678-815f-fdb7744c3b56 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.447014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.448484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.448983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.449483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.450502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:32.451278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.452924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.455625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.461342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee98e6-b5ea-43fd-9016-86a77fd9d3f3 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.466525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 445/887] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:32.474393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.501529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.502261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.502612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.503006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.504618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:32.505911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.507237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.532463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 443/888] 10.222.0.72 () {56 vars in 1149 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:32.533443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.604456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff7d70-5223-458e-ac71-a2a84dd1231e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.639610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.640244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.640566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.640843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.641474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:32.641776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.643176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574fc8b-1b60-4bd6-aeaa-e6967801bc7d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.649980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 446/889] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:32.657565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4b16c-9d87-450c-9749-96040713f4ea None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.699677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.700362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.700644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.700916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.726696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.788527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.789997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.850405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc19e-51dd-4d38-92a1-4a83de1d33ec None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.850736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc19e-51dd-4d38-92a1-4a83de1d33ec None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.851007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc19e-51dd-4d38-92a1-4a83de1d33ec None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.853953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 447/890] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:32.858474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.858749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.859049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.884619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:32.885830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:32.911260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.921095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 444/891] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:32.927629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.974388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 448/892] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:32.984268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:33.036417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d8fa8-1c70-45fd-ac71-2d1d2b067024 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.074075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.074729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.075040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.075315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.086824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.088300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:33.103150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:33.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:33.105419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:33.105419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:33.105419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:33.105419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 445/893] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:33.111936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.217190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1ab4d-1a8d-42ed-a868-c97704469725 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.257131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.257747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.258025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.258293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.258959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:33.259264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.260636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.266647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.273462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 449/894] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:33.278572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.299039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:33.377500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c7e1-0ae0-46d9-9f0b-f6f17d33ec9d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.379918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:33.406505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043bc1-b2b9-436d-a806-7611fc4f483f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.421278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.422171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.422971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.423259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.424134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.426156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:33.436939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:33.437999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:33.438891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 446/895] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:33.448199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.449356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.450079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.450429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.450769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.461276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.463527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:33.479854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:33.481783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:33.482937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 450/896] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:33.489978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.567589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb30a4a-b53f-45b8-9c57-9aadafac1b15 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.592551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:33.612393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.613709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.614078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.614444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.615246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:33.615648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.618088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.624155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b099d6b-5512-49fa-90d7-ea4b92c6007a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.626935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 447/897] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:33.631420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.669110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:33.672798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.673412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.673692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.673964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.674808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:33.675147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.678812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.695015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 451/898] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:33.700488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.767417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db1553-67f3-451e-a9d9-8e6a8c645722 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.809610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.810234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.810506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.810829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.811659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:33.811963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.813907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.825063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 448/899] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:33.825217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9587bff-0a53-493a-8419-0a85f64d74e6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.826597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:33.867525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.868141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.868424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.868697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.869485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] RBAC: Adding query filter params (role.id=f0863cec30b84a909f64bc2f92c4a947, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1de6964e61bd46eca46e1db919ae14c3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:33.869792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.871414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:33.883023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 452/900] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/role_assignments?user.id=1de6964e61bd46eca46e1db919ae14c3&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=f0863cec30b84a909f64bc2f92c4a947 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:33.941473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:33.950522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9021b-a5cc-4e9f-a273-405bcd13b055 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:33.992460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:33.993088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:33.993368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:33.993638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:33.994348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] RBAC: Adding query filter params (interface=public, service_id=f49b3c96e0a64cebb8ffa408c7df1aba, region_id=RegionOne) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:33.994650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:33.996050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787041c-9c6a-480c-a55b-1b2cbf185d93 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:34.006972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 449/901] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:33 2022] GET /identity/v3/endpoints?service_id=f49b3c96e0a64cebb8ffa408c7df1aba&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:34.012079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:34.015235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:34.122955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cadd9a-8101-4a1a-bc6b-1e3a90dc3492 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:34.158351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcd778-9dce-42fb-815f-210e1e3c5a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:34.158987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcd778-9dce-42fb-815f-210e1e3c5a00 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:34.159267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcd778-9dce-42fb-815f-210e1e3c5a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:34.159537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcd778-9dce-42fb-815f-210e1e3c5a00 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:34.160984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcd778-9dce-42fb-815f-210e1e3c5a00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:34.162540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcd778-9dce-42fb-815f-210e1e3c5a00 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:34.174997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 453/902] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:34 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:46:34.439662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5975536-1525-4c5c-83e9-a6640b936a21 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:34.440150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5975536-1525-4c5c-83e9-a6640b936a21 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:34.440150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5975536-1525-4c5c-83e9-a6640b936a21 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:34.442522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 450/903] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:34.448317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:34.448836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:34.448993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:34.467295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:34.468497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:34.532020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:34.540692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 454/904] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:34.820813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5975536-1525-4c5c-83e9-a6640b936a21 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:34.935817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5975536-1525-4c5c-83e9-a6640b936a21 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:34.971817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:34.972637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:34.973025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:34.973281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:34.973867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:34.975359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-ddbfe59c-35f2-4276-8dcd-7fab0a3b7cef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:34.984463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:34.985392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:34.986305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 451/905] 10.222.0.72 () {56 vars in 1132 bytes} [Mon Sep 26 17:46:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:34.994447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.022223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e0b3-edd3-4fa5-8434-7cac073b44c6 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.022547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e0b3-edd3-4fa5-8434-7cac073b44c6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.022788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e0b3-edd3-4fa5-8434-7cac073b44c6 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.024448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 452/906] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:35.029607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.029897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.030182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.049858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:35.051582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:35.071144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:35.116052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97025a6b-f0b8-4971-87ad-ca045fc84612 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.135102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 453/907] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:35.158333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.159252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.160304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.160724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.161541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:35.162021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.163673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.169297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 455/908] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:35.175061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.281096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7b7f5-fbc1-47a5-9c44-964336be3ec9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.315659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.316850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.317134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.317409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.318039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:35.318345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.319998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.325323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 454/909] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:35.330954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.393435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:35.406377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.427206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:35.443386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b2c4b-967c-495f-a448-0383bc379eeb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.482218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.483618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.484128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.484626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.485606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] RBAC: Adding query filter params (interface=public, service_id=eb085e1cee774f09be37c68c7ea31239, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:35.486103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.487688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.493524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 456/910] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:35 2022] GET /identity/v3/endpoints?service_id=eb085e1cee774f09be37c68c7ea31239&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:35.499346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.526117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b590e3-fe6f-4e6a-88b3-79b46a38ad62 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.563311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9255e4-f320-4847-b132-ac63469b7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.564219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9255e4-f320-4847-b132-ac63469b7d0d None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.564719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9255e4-f320-4847-b132-ac63469b7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.565174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9255e4-f320-4847-b132-ac63469b7d0d None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.566661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9255e4-f320-4847-b132-ac63469b7d0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.568241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9255e4-f320-4847-b132-ac63469b7d0d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.593071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 455/911] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:35 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:35.630976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f401f6b-817c-4e37-a9e3-bd164e2c78ea None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.663824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:35.685317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2e780-99c0-4fbc-898d-ca1964a7eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.685936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2e780-99c0-4fbc-898d-ca1964a7eab0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.686216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2e780-99c0-4fbc-898d-ca1964a7eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.686486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2e780-99c0-4fbc-898d-ca1964a7eab0 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.687126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2e780-99c0-4fbc-898d-ca1964a7eab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.688516 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2e780-99c0-4fbc-898d-ca1964a7eab0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.696408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 457/912] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:35 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:46:36.143552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dc179-3978-415b-8f70-da63d1bb1347 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:36.143928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dc179-3978-415b-8f70-da63d1bb1347 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:36.144306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dc179-3978-415b-8f70-da63d1bb1347 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:36.145975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 456/913] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:36.151410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e832fea-0cb7-4d83-86bf-97c70dc28937 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:36.151694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e832fea-0cb7-4d83-86bf-97c70dc28937 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:36.151970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e832fea-0cb7-4d83-86bf-97c70dc28937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:36.171598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3e832fea-0cb7-4d83-86bf-97c70dc28937 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:36.172774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3e832fea-0cb7-4d83-86bf-97c70dc28937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:36.245918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 458/914] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:36.519674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dc179-3978-415b-8f70-da63d1bb1347 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:36.543306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4833-06e6-4ad9-8b82-ff86b42ef12f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:36.543844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4833-06e6-4ad9-8b82-ff86b42ef12f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:36.543844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d4833-06e6-4ad9-8b82-ff86b42ef12f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:36.545449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 459/915] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:36.550333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:36.550625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:36.550948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:36.569588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:36.570798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:36.610454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:36.651274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dc179-3978-415b-8f70-da63d1bb1347 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:36.654928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 460/916] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:36.694372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:36.695065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:36.695341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:36.695608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:36.706103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:36.712127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:36.713063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:36.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 457/917] 10.222.0.72 () {56 vars in 1120 bytes} [Mon Sep 26 17:46:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:36.719334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:36.769659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:36.829239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385864a-a5fa-4b02-a1c0-9cb1318860ed None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:36.865105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:36.865723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:36.866006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:36.866281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:36.866974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:36.867282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:36.868648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:36.877125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 461/918] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:46:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:36.882635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:36.959547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:36.995710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf88a11-9a25-4a9f-a9b2-0efbcac83e7c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.044791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.045436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.045741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.046265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.060823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.062808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.069087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852cdee-a22f-4806-b311-eb5f97004bdd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.075184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.076760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:37.082541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:37.083524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:37.089154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:37.089154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:37.089154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:37.089154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 458/919] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:37.090979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.122179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.123805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.124300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.125039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.126209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.128060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:37.140282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:37.141278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:46:37.142120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:37.147100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 462/920] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:37.152224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.205574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1be43-a298-4b4f-9d4d-f31277b52469 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.240918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.241577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.241870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.242147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.242862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:37.243179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.244544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.257171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 459/921] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:37.263441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5afaaf-13ac-48c2-9f75-741482e886c1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.263567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.302187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.303219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.303713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.304230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.305155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:37.305758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.307392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.318381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 463/922] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:37.320141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.320623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.360203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.372154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7160b-ceb5-44a8-9a5d-29e9b84b3b9f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.408617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.409248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.409662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.410446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.418373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.419855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:37.433179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:37.434176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:37.435221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:37.439710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 460/923] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:37.440613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.443303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44c7c-5a8b-452b-b4d5-51cbb7c7a3b7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.482951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.483935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.484594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.485061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.485964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:37.486477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.488958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.501538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 464/924] 10.222.0.72 () {56 vars in 1114 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:37.503156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.559111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b02520-9833-4227-bf6a-87ab1b2c489c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.569558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.596788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.598143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.598530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.598922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.599617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:37.599923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.601354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.613376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.618622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded31ca1-aadb-41b3-84fd-559ed7a483f4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.625250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 461/925] 10.222.0.72 () {56 vars in 1149 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:37.627012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.658821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.660648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.661149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.661635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.662672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.664488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.695359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 465/926] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:37.745372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b24b31-5b2c-4c41-85de-2fc5b32726f5 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.795899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.796548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.796829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.797102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.809350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.810806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-07511bbc-4df2-4550-8f33-9f530eab64e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:37.829096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:37.830110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:37.831545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:37.831545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:37.831545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:37.831545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 462/927] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:37.836947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.955791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550185-d89f-4b32-a97d-570d90a207a2 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.956103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550185-d89f-4b32-a97d-570d90a207a2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.956356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c550185-d89f-4b32-a97d-570d90a207a2 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.958217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 463/928] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:37.963580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e339119-586c-42d4-85ee-393771b50f64 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.963728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0796c-64ca-426a-9edb-70bb373488f1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.963935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e339119-586c-42d4-85ee-393771b50f64 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.964134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e339119-586c-42d4-85ee-393771b50f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.989045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2e339119-586c-42d4-85ee-393771b50f64 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:37.990276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2e339119-586c-42d4-85ee-393771b50f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:38.007420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.010524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.011171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.011461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.011733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.012381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:38.012688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.014068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980564b6-3456-4153-94e9-1e029d91def3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.026032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 466/929] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:38.027355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae466-c771-48cf-ac59-0217c5d9ee3c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.027640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae466-c771-48cf-ac59-0217c5d9ee3c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.027911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae466-c771-48cf-ac59-0217c5d9ee3c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.029320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 467/930] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:38.031570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae466-c771-48cf-ac59-0217c5d9ee3c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.051682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.073423 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 464/931] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:38.075477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.075963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.076448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.098569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:38.100078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:38.151090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae466-c771-48cf-ac59-0217c5d9ee3c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.175890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 465/932] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:38.190457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.191872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.192297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.192747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.200356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.202065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:38.214481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:38.215579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:38.216677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.218641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 468/933] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:38.227934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.305629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.333407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb3dfb-649f-4834-8c3b-f3cc1d3675b0 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.369447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.370320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.370758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.371824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.372806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.373506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:38.373810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.375654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.392700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 466/934] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:38.399043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.457788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.483895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a491db0-2796-43d9-b013-75eaed5a82b7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.505770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e4c5c-6892-406c-92e8-45471284d723 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.522248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.523370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.523939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.524426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.525213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.526798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:38.536563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:38.537545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:38.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.540351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 469/935] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:38.543684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.547791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.549355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.549643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.549924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1de6964e61bd46eca46e1db919ae14c3/roles/f0863cec30b84a909f64bc2f92c4a947` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.552793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=1de6964e61bd46eca46e1db919ae14c3, role_id=f0863cec30b84a909f64bc2f92c4a947)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.554700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.557849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.572581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 467/936] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:38 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1de6964e61bd46eca46e1db919ae14c3/roles/f0863cec30b84a909f64bc2f92c4a947 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:46:38.573624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.669457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6a1c0-0cc8-4d50-8c50-953b153e99f2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.718226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa7d6e-c7a9-45d6-af07-c0c05324ba4a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.720335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.722976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.723458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.723948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.724740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.726575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:38.740978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:38.742274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:46:38.743432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:38.746265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 470/937] 10.222.0.72 () {56 vars in 1132 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:38.753670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.758943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.773670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.775066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.775707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.776335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.777169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:38.777493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.780514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.788897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 468/938] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:38.795117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.860977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.879797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c0188-b3c1-4084-b022-c8c910a12107 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.915472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71abd11-3c90-45c3-85c5-1577fb167bf6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.916644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.917258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.917542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.917819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.918445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:38.918753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.920157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.925999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 471/939] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:38.935800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.963957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.965437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.965897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.966321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.967435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:38.967922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.969708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.978277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 469/940] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:38.984802 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:39.028466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:39.075145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acb32d-1c6c-4d67-b834-c9ed7b59be47 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:39.111521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:39.112548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:39.113620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:39.114096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:39.114954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:39.115463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:39.117153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:39.120980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4045-7c61-4188-a4c6-9e4d16e44cf8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:39.127644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 472/941] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:39.129614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:39.150505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:39.167072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:39.167720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:39.168029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:39.168421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:39.169898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] RBAC: Adding query filter params (interface=public, service_id=2c93353d4abe4f2fa64e9cc183398a70, region_id=RegionOne) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:39.170234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:39.171655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:39.179571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 470/942] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:38 2022] GET /identity/v3/endpoints?service_id=2c93353d4abe4f2fa64e9cc183398a70&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:39.188130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:39.245921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6816018-4f26-45c0-b406-b27d66f17fb1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:39.280322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:39.284015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa05cf-e5b6-4c80-ab4d-e09e57779023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:39.284982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa05cf-e5b6-4c80-ab4d-e09e57779023 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:39.285275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa05cf-e5b6-4c80-ab4d-e09e57779023 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:39.285716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa05cf-e5b6-4c80-ab4d-e09e57779023 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:39.286418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa05cf-e5b6-4c80-ab4d-e09e57779023 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:39.288055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa05cf-e5b6-4c80-ab4d-e09e57779023 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:39.293524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533b9da-5a5d-4778-b55e-211d1b0c0010 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:39.331373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 473/943] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:39.352392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa8c6b-493b-4e92-8259-f2c106403ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:39.353082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa8c6b-493b-4e92-8259-f2c106403ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:39.353363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa8c6b-493b-4e92-8259-f2c106403ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:39.353633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa8c6b-493b-4e92-8259-f2c106403ec0 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:39.354296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa8c6b-493b-4e92-8259-f2c106403ec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:39.355737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa8c6b-493b-4e92-8259-f2c106403ec0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:39.364240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 471/944] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:39 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:46:40.046707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4a832-5dd8-476f-a86a-a8287f32deb5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.047253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4a832-5dd8-476f-a86a-a8287f32deb5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.047253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4a832-5dd8-476f-a86a-a8287f32deb5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.048828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 474/945] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:40.054910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.055334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.055464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.074157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:40.075372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:40.123551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:40.151506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 472/946] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:40.420179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4a832-5dd8-476f-a86a-a8287f32deb5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:40.532457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4a832-5dd8-476f-a86a-a8287f32deb5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:40.569486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:40.570435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.570939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.571413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.572249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:40.573866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:40.576775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING keystone.server.flask.common [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 17:46:40.599536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.sql.core [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 17:46:40.599536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 17:46:40.599536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [parameters: {'id': 'b4040fff384a487e990b8d7611a5fc56', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 17:46:40.599536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81410) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b4040fff384a487e990b8d7611a5fc56.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b4040fff384a487e990b8d7611a5fc56. Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:46:40.606891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 26 17:46:40.607857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application yield resource Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise self.value Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application session.commit() Sep 26 17:46:40.609047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.flush() Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application compat.raise_( Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 17:46:40.610176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 17:46:40.611246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:46:40.612086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [parameters: {'id': 'b4040fff384a487e990b8d7611a5fc56', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:40.613675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:40.614576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b4040fff384a487e990b8d7611a5fc56. Sep 26 17:46:40.615544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:40.617783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 475/947] 10.222.0.72 () {60 vars in 1140 bytes} [Mon Sep 26 17:46:40 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:40.623437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:40.729244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1ce2c-157b-414d-864d-8295886a88f9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:40.766195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:40.767158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.767654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.768116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.775951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:40.777648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-4ded5699-3a22-4f85-bfec-70807af6841c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:40.789582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:40.790508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:40.791460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:40.793533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 473/948] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:40.804537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:40.854866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:40.919561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e210b-1421-4270-b63f-deb5b5561b45 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:40.923645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544de49-a6d2-419c-b6c1-2b69f7419a15 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.923929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544de49-a6d2-419c-b6c1-2b69f7419a15 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.924201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544de49-a6d2-419c-b6c1-2b69f7419a15 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.927712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 474/949] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:40.931801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.932084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.932360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.952486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:40.953712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:40.969511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:40.970127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.970409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.970682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.971382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:40.971687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:40.973559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:40.995054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 476/950] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:41.002879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.054950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 475/951] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:41.321761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.427011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99965b41-8f45-4e3a-b658-058639b658ac None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.428972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.464535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d748e0-dce9-4542-a2d3-292ebafafbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.465182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d748e0-dce9-4542-a2d3-292ebafafbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.465464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d748e0-dce9-4542-a2d3-292ebafafbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.465740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d748e0-dce9-4542-a2d3-292ebafafbe4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.477134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d748e0-dce9-4542-a2d3-292ebafafbe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-88d748e0-dce9-4542-a2d3-292ebafafbe4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:41.482162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:41.483251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:41.484524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 477/952] 10.222.0.72 () {56 vars in 1120 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:41.492792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.610818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf76859-dbc6-45a9-a611-7a070a7e7557 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.624580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842e60-a259-41e9-ad08-de9600e7f258 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.624580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842e60-a259-41e9-ad08-de9600e7f258 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.624580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842e60-a259-41e9-ad08-de9600e7f258 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.625403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 478/953] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:41.630709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc8ad-e41a-4524-91f7-efea07ec228b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.631018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc8ad-e41a-4524-91f7-efea07ec228b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.631297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc8ad-e41a-4524-91f7-efea07ec228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.650209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-145fc8ad-e41a-4524-91f7-efea07ec228b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:41.651784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-145fc8ad-e41a-4524-91f7-efea07ec228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:41.652150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.652768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.653051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.653327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.654200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:41.655365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.656748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.669070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 476/954] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:41.674800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.707772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.729000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 479/955] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:41.730482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f12ae-0928-4c7f-9149-9da50b439e25 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.731136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f12ae-0928-4c7f-9149-9da50b439e25 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.731589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f12ae-0928-4c7f-9149-9da50b439e25 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.733320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 480/956] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:41.745031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878a820-ae44-4411-a601-b215827e1192 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.745701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878a820-ae44-4411-a601-b215827e1192 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.746180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878a820-ae44-4411-a601-b215827e1192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.778819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4878a820-ae44-4411-a601-b215827e1192 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:41.780891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4878a820-ae44-4411-a601-b215827e1192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:41.786284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef782d8d-273f-405a-af91-521bcb0c631b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.823757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.824390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.824668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.824944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.836289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.837767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:41.853149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:41.854125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:41.855060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:41.855060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:41.855060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:41.858996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 477/957] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:41.860189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.879223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 481/958] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:41.947707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.966165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a8935-6155-4447-b0f5-b38b1109098f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.004035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.004793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.005075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.005349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.006004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:42.006308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.007696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.025424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 478/959] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:42.026781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878a820-ae44-4411-a601-b215827e1192 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.047210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.053101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.152790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878a820-ae44-4411-a601-b215827e1192 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.168194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751e81e-9726-4b20-92b3-9fca8b562bf0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.196166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.196795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.197070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.197341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.204399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.205824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.207259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.208059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.208823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.209107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.209698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.211073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:42.217402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:42.218390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 482/960] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:42.224415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:42.225114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:42.228513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:42.232480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 479/961] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:42.234807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.237231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.313975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.333576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063136c-7803-4a8e-8511-cc3d0fbe920d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.336099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be143ca1-d2c0-492c-b3a2-5fcc43b2a1bc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.372357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.372569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.372983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.373104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.373278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.373388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.373535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.373625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.374146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:42.374241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.374447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.375538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.375829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.380989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 480/962] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:42.384907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:42.385886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:42.386730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 483/963] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:42.388071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.388153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.461102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.496670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7af640-74d7-4297-bfdb-2fd0fe3586ff None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.498060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e985a-f196-4369-b767-091bc82346c1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.533848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.535465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.535899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.536323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.536451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.536934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.537217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.537493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.537820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:42.538116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:42.538422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.538586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.539810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.540328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.544898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 484/964] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:42.545676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.561439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 481/965] 10.222.0.72 () {56 vars in 1149 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:42.562462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.567068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.656359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e354-e4d3-4bff-9db5-9fa4a84d154d None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.681795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d804e64-1f40-432e-b0b5-869d2820fa05 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.693341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.694218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.694500 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.694776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.695435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:42.695740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.697105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.701864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 485/966] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:42.702652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.716903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.727163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.727773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.728054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.728328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.729024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.730383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.759235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 482/967] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:42.760284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:42.806278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:42.841763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0338c-ba90-48f9-bf94-b26abef3b814 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.890276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8c71e-2f13-42d0-942f-75102da38e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.891340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8c71e-2f13-42d0-942f-75102da38e35 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.891897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8c71e-2f13-42d0-942f-75102da38e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.892518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8c71e-2f13-42d0-942f-75102da38e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.894516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15887d7-35c5-4630-a9e6-d7ac48fcff82 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:42.934932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:42.935667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:42.936103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:42.936575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:42.937395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:42.937881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.939475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:42.945867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 483/968] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:42.993388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8c71e-2f13-42d0-942f-75102da38e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:42.995299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8c71e-2f13-42d0-942f-75102da38e35 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:43.077770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:43.122460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 486/969] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:43.123880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.179213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:43.243107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e49e40-4679-4fee-a8ff-3d0c986aaea9 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:43.279575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:43.280496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.280944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.281366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.293759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:43.295484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:43.311535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:43.312788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:43.313924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:43.313924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:43.313924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:43.314135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db56e5-3482-498c-b924-f33b4c7cb228 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.314135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db56e5-3482-498c-b924-f33b4c7cb228 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.314135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db56e5-3482-498c-b924-f33b4c7cb228 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.314568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 487/970] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:43.316102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 484/971] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:43.320050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.320543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.320999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.321985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.346783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:43.349073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:43.423539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:43.433161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39d540-2d2f-4077-8390-e27432f1bff8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:43.447654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 488/972] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:43.480368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:43.480973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.481251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.481521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.482156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:43.482455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:43.483910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:43.497344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 485/973] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:43.502902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.619848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:43.623485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c4b92-4fd0-49b7-9b65-8baf70b36a15 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:43.663368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:43.664225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.664509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.664784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.672125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:43.673573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:43.686261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:43.688398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:43.690517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:43.694868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 489/974] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:43.695177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.734546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.813360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e97bc-a305-471a-b98b-de8979c1d9a1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:43.854743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaca2d5-ac60-4a5a-9e2f-2f4f272692a3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:43.859257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:43.859875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.860158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.860430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.861126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:43.861434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:43.863051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:43.876798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 486/975] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:43.884734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.901771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:43.902754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.903100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.903380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.904317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:43.905906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:43.907090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:43.927353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.sql.core [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 17:46:43.927353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 17:46:43.927353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [parameters: {'user_id': 'dd69eba7a0d14e99a286d12c9a3a1431', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 17:46:43.927353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81410) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 26 17:46:43.936843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:46:43.937758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application yield resource Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise self.value Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application session.commit() Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 17:46:43.939429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.session.flush() Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application compat.raise_( Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 17:46:43.940384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:46:43.942003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application result.read() Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:46:43.943052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dd69eba7a0d14e99a286d12c9a3a1431', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:43.945937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 26 17:46:43.948025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 490/976] 10.222.0.72 () {60 vars in 1131 bytes} [Mon Sep 26 17:46:43 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:43.950205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:43.982458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:44.037738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b86a5-137c-4bed-816b-dfbf65705dc8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.068040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd126d44-fadf-4d8c-b6ad-e216b64401cc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.077896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.078871 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.079358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.079831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.080768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] RBAC: Adding query filter params (role.id=f0863cec30b84a909f64bc2f92c4a947, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1de6964e61bd46eca46e1db919ae14c3) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:44.082114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:44.084572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:44.099113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 487/977] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:43 2022] GET /identity/v3/role_assignments?user.id=1de6964e61bd46eca46e1db919ae14c3&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=f0863cec30b84a909f64bc2f92c4a947 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:44.113223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.113893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.114185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.114505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.124246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:44.125961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-8d163541-e707-4d02-95bd-9dc3761b7941 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:44.141162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:44.142202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:44.143579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 491/978] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:44.148853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:44.239642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:44.264954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f8b01-5111-4b05-8d70-70a6565d6beb None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.302795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.303493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.303780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.304052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.304761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:44.305066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:44.306512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c7338-6163-4ed5-8e2d-3655ded86b74 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:44.347557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 488/979] 10.222.0.72 () {56 vars in 1101 bytes} [Mon Sep 26 17:46:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:45.125821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca1045-71c5-41be-b17a-af34ff8d6ba7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.126357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca1045-71c5-41be-b17a-af34ff8d6ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.126357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca1045-71c5-41be-b17a-af34ff8d6ba7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.131134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 492/980] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:45.136321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.136616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.136896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.156970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:45.158771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:45.209059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee71c4-d5d7-4c31-98a4-7173a1ed2fb0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.209370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee71c4-d5d7-4c31-98a4-7173a1ed2fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.209615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee71c4-d5d7-4c31-98a4-7173a1ed2fb0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.211245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 493/981] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:45.216103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.216386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.216656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.234955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 489/982] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:45.235138 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:45.236396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:45.304643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 494/983] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:45.510461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:45.572747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:45.586072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:45.589457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:45.629112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79353c1b-783b-43f5-9192-bd5a82038b51 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:45.666111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:45.666747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.667048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.667318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.667895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:45.669247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:45.679159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:45.680360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:45.681397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:45.687176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 490/984] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:45.687729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6614ca-c2cb-4458-b950-8cc9a91ff070 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:45.687915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:45.728755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:45.729367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.729644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.729917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.730571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:45.731973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:45.749323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 495/985] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:45 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:45.799068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785320a7-1419-4468-91ef-ef4442b95179 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:45.833899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:45.850494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:45.851148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.851433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.851709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.852354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:45.852663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:45.854134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:45.866016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 491/986] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:45.867939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:45.946377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:45.986735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfd6f-5d4f-488a-9f9d-a25134bfba80 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:46.021426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:46.022042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:46.022329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:46.022606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:46.023270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:46.023577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:46.024951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:46.034947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 496/987] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:46.037015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:46.139397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aea41-da2c-41de-9ceb-e614d9adba1b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:46.176938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:46.177554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:46.177835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:46.178108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:46.178819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] RBAC: Adding query filter params (interface=admin, service_id=2c93353d4abe4f2fa64e9cc183398a70, region_id=RegionOne) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:46.179156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:46.180516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6f926-0ca0-42b7-b643-b28fb6cbee00 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:46.190069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 492/988] 10.222.0.72 () {56 vars in 1249 bytes} [Mon Sep 26 17:46:46 2022] GET /identity/v3/endpoints?service_id=2c93353d4abe4f2fa64e9cc183398a70&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:46.191604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:46.294477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cb70-fd5c-4def-9727-f59f696e2503 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:46.332811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb67d6a-6bf1-4349-b5b9-c256c24ed7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:46.333425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb67d6a-6bf1-4349-b5b9-c256c24ed7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:46.333708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb67d6a-6bf1-4349-b5b9-c256c24ed7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:46.333988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb67d6a-6bf1-4349-b5b9-c256c24ed7f2 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:46.334593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb67d6a-6bf1-4349-b5b9-c256c24ed7f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:46.335998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb67d6a-6bf1-4349-b5b9-c256c24ed7f2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:46.337820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:46.351300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 497/989] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:46 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:46:46.415791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0892-635e-4e99-b2ab-771af8f53a25 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:46.415990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0892-635e-4e99-b2ab-771af8f53a25 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:46.416258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0892-635e-4e99-b2ab-771af8f53a25 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:46.417827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 493/990] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:46.423471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:46.423787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:46.424036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:46.444578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:46.445794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:46.532468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 498/991] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:46.801213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0892-635e-4e99-b2ab-771af8f53a25 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:46.860426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:46.908219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0892-635e-4e99-b2ab-771af8f53a25 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:46.948854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:46.949552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:46.949835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:46.950112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] PATH_INFO: `/v3/services/image` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:46.950704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:46.952113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:46.965006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:46.966044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:46.971193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 494/992] 10.222.0.72 () {56 vars in 1105 bytes} [Mon Sep 26 17:46:46 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:46.973610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:47.078232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdc56a-5897-45d0-8dc4-216d2789a6cd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:47.109252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:47.116788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:47.117399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:47.117679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:47.117951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:47.118584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] RBAC: Adding query filter params (name=image) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:47.118921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:47.120290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c689f5a-ba88-4d9d-b650-fd12587c1a68 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:47.129938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 499/993] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:47.130969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:47.234140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440920e9-dcf7-460b-9225-8038edb7275d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:47.252376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:47.274382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:47.275018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:47.275444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:47.275728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:47.276357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] RBAC: Adding query filter params (type=image) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:47.276659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:47.278009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff4f4b-45d1-4e13-b751-c08992e10c9c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:47.283738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 495/994] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:48.105400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4583f-2597-420e-8761-9391d09dba4a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.105921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4583f-2597-420e-8761-9391d09dba4a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.106034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4583f-2597-420e-8761-9391d09dba4a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.107493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 500/995] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:48.112527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ee04-940e-41c2-b1b0-e5482dcbe83d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.113482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ee04-940e-41c2-b1b0-e5482dcbe83d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.113644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ee04-940e-41c2-b1b0-e5482dcbe83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.131866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e922ee04-940e-41c2-b1b0-e5482dcbe83d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:48.133073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e922ee04-940e-41c2-b1b0-e5482dcbe83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:48.205992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 496/996] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:48.479180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4583f-2597-420e-8761-9391d09dba4a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.578745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e833-d430-4a4d-bbdc-7fee25f2c7d8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.579112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e833-d430-4a4d-bbdc-7fee25f2c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.579350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e833-d430-4a4d-bbdc-7fee25f2c7d8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.580922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 497/997] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:48.586669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.586985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.587266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.587816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4583f-2597-420e-8761-9391d09dba4a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.605920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:48.607159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:48.624807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.625498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.625783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.626068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.626697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.628121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:48.637142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:48.638130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:48.639031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 501/998] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:48.644584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.690864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 498/999] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:48.705672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:48.750139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa8c4f-9b1f-45e2-af2a-c86e1a737e0a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.783508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.784118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.784921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.785199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.785903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:48.786226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.787730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.797141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 502/1000] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:48.798200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.841421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:48.903167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e5eef-53a4-4777-abff-518f8a40b044 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.939357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.939977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.940258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.940532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.941490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:48.941910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.943400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.949004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 499/1001] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:48.954723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.954979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:49.064251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba58bd-b052-4af9-a0c8-e4f4a2a068fa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:49.071081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df30574-af99-4787-9059-c2c73d5d46c1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:49.079464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:49.088541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:49.102274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:49.102913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.103193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.103470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.104171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] RBAC: Adding query filter params (interface=public, service_id=269465e5e58d4166851ed1c8e0fceffb, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:49.104475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:49.105834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:49.110937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 503/1002] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/endpoints?service_id=269465e5e58d4166851ed1c8e0fceffb&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:49.112679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:49.113315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.113598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.113873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.114460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:49.115867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:49.116538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:49.126744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:49.127773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:46:49.128974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:49.131134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 500/1003] 10.222.0.72 () {56 vars in 1126 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:49.134200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:49.225615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f64ebe-cb64-41d9-a7a7-dd3aa4aeb947 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:49.247839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946674d6-c4bc-413b-aa48-80002bde820e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:49.259719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:49.260327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.260608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.260881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.261477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:49.262867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:49.269843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 504/1004] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:49 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:46:49.287426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:49.288366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.288881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.289305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.290111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:49.290608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:49.292204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:49.298462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 501/1005] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:49.305144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:49.364062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:49.425735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3c81a-dd02-4435-8114-f239506198ca None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:49.464623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:49.465242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.465526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.465802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.466441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:49.466750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:49.468147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd232419-0b68-473f-885d-5a905d87e454 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:49.473915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 505/1006] 10.222.0.72 () {56 vars in 1122 bytes} [Mon Sep 26 17:46:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:49.479692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:49.492228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994a906-5d2d-4661-9595-b9ddab32a288 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.492512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994a906-5d2d-4661-9595-b9ddab32a288 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.492784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994a906-5d2d-4661-9595-b9ddab32a288 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.494183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 506/1007] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:49.499674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.499961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.500236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.518141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:49.521259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:49.522498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:49.599700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb669f4-c690-455f-a632-b4a1c94a0a3d None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:49.603177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 507/1008] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:49.640470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5cb5d-9454-4daf-b9df-4ff7c9d6393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:49.641005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5cb5d-9454-4daf-b9df-4ff7c9d6393b None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:49.641284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5cb5d-9454-4daf-b9df-4ff7c9d6393b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:49.641557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5cb5d-9454-4daf-b9df-4ff7c9d6393b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:49.642230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5cb5d-9454-4daf-b9df-4ff7c9d6393b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:49.643623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5cb5d-9454-4daf-b9df-4ff7c9d6393b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:49.666864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 502/1009] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:49.881421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:49.934410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:49.997257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2077-ee7d-45e3-bb7d-ea3e260acbb9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:50.040317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f584f43-9fea-4b49-a87c-0ee0cb9582e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:50.040930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f584f43-9fea-4b49-a87c-0ee0cb9582e0 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:50.041210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f584f43-9fea-4b49-a87c-0ee0cb9582e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:50.041481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f584f43-9fea-4b49-a87c-0ee0cb9582e0 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:50.042137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f584f43-9fea-4b49-a87c-0ee0cb9582e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:50.043527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f584f43-9fea-4b49-a87c-0ee0cb9582e0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:50.058620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 508/1010] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:49 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:51.539408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c8f2-b087-4af3-8399-db42547b49d5 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:51.539940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c8f2-b087-4af3-8399-db42547b49d5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:51.540209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481c8f2-b087-4af3-8399-db42547b49d5 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:51.542350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 503/1011] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:51.555576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d284ad7-887f-4cea-97b9-7146a658a1a7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:51.559713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d284ad7-887f-4cea-97b9-7146a658a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:51.559993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d284ad7-887f-4cea-97b9-7146a658a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:51.584264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3d284ad7-887f-4cea-97b9-7146a658a1a7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:51.585472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d284ad7-887f-4cea-97b9-7146a658a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:51.671861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 509/1012] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:51.850325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11e7b6-be62-4fe5-84e3-d0a4f2485c22 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:51.850615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11e7b6-be62-4fe5-84e3-d0a4f2485c22 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:51.850802 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11e7b6-be62-4fe5-84e3-d0a4f2485c22 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:51.852316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 504/1013] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:51.857400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:51.857684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:51.857950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:51.876806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:51.877975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:51.900221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:51.958879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11e7b6-be62-4fe5-84e3-d0a4f2485c22 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:51.959080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 510/1014] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:52.067618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:52.076454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11e7b6-be62-4fe5-84e3-d0a4f2485c22 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.112617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.113169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.113450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.113721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] PATH_INFO: `/v3/services/compute` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.114337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.115775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:52.124809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:52.125810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:46:52.126666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:52.131063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 505/1015] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:46:52.131927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:52.230903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:52.242738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fce6c-47cc-48aa-bd29-1c9d9c6dbd5c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.278826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.279471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.279747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.280019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.280654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:52.280958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.282323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.289339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 511/1016] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:52.294707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:52.318002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:52.342941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eae0f8-9b3c-4270-8c43-6a9cefc057ee None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.379221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84a73e-f413-4ad7-89b5-5a18858b0a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.379762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84a73e-f413-4ad7-89b5-5a18858b0a04 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.380042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84a73e-f413-4ad7-89b5-5a18858b0a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.380313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84a73e-f413-4ad7-89b5-5a18858b0a04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.381422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84a73e-f413-4ad7-89b5-5a18858b0a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.382857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84a73e-f413-4ad7-89b5-5a18858b0a04 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.384431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 506/1017] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:52.385546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d99dd-175b-4707-acb0-4fc5b4a5e5e1 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.385826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d99dd-175b-4707-acb0-4fc5b4a5e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.386098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d99dd-175b-4707-acb0-4fc5b4a5e5e1 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.387317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 507/1018] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:52.390122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d99dd-175b-4707-acb0-4fc5b4a5e5e1 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:52.406896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37785a87-5054-4463-8b7d-e9bbef43eb63 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.418625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:52.443039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.443636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.444181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.444546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.445077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:52.445379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.446738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc0251-d8c4-422e-8f75-d66b9aad4c03 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.451630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 512/1019] 10.222.0.72 () {56 vars in 1112 bytes} [Mon Sep 26 17:46:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:52.452599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.452884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.453157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.471525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:52.472733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:52.499327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d99dd-175b-4707-acb0-4fc5b4a5e5e1 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.533982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.534616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.534927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.535214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.535861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.537289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.539566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 513/1020] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:52 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:52.540545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:52.558896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:52.582065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 508/1021] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:52 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:52.673533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c02d3-d52f-4c2f-b4da-62fa3b61bc98 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.712608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.713586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.714029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.714483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.715365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.716928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.739507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 514/1022] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:52.816422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:52.902544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:52.927299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef3bb-e341-4356-8f88-97d17e7c541a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:52.969869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:52.970511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:52.970790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:52.971092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] PATH_INFO: `/v3/services/image` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:52.971672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:52.973076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:52.984279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:52.985283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:52.986135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 509/1023] 10.222.0.72 () {56 vars in 1105 bytes} [Mon Sep 26 17:46:52 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:52.992340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:53.067212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:53.103815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a86597-db2e-41d7-ba2d-6c69bbddb025 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:53.150643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:53.151303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:53.151591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:53.151863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:53.152524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:53.152918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:53.154293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:53.165369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 515/1024] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:53.166379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:53.271504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96cb5f-6b15-4d8a-95e9-544ab44e91cd None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:53.305682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:53.306297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:53.306576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:53.306873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:53.307504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] RBAC: Adding query filter params (type=image) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:53.307807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:53.309160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:53.319945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 510/1025] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:53.321279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:53.429462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6523f1e-affd-4f64-a058-21aa4dc985d3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:53.465233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:53.465842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:53.466126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:53.466398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:53.467113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] RBAC: Adding query filter params (interface=public, service_id=c2ffa4af30ed4b0e93ea8de38f6c7c94, region_id=RegionOne) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:53.467424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:53.468772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740975d-14c5-4b74-a3e8-cf90c1f4d28a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:53.470216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:53.479921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 516/1026] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:46:53 2022] GET /identity/v3/endpoints?service_id=c2ffa4af30ed4b0e93ea8de38f6c7c94&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:53.481042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:53.484131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:53.591281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473221-e47a-4410-8a51-074441edcddd None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:53.626388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188870a-2ed5-4946-8566-c935ad5bf03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:53.627033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188870a-2ed5-4946-8566-c935ad5bf03c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:53.627318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188870a-2ed5-4946-8566-c935ad5bf03c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:53.627592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188870a-2ed5-4946-8566-c935ad5bf03c None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:53.628866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188870a-2ed5-4946-8566-c935ad5bf03c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:53.630225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188870a-2ed5-4946-8566-c935ad5bf03c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:53.637108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 511/1027] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:46:53 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:46:54.864984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9b470-fd92-4b79-9da2-c5cb37886a06 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:54.865541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9b470-fd92-4b79-9da2-c5cb37886a06 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:54.865541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9b470-fd92-4b79-9da2-c5cb37886a06 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:54.867068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 517/1028] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:54.872604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bfca9-532f-4164-89f9-2e6724e22c7d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:54.873020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bfca9-532f-4164-89f9-2e6724e22c7d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:54.873206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bfca9-532f-4164-89f9-2e6724e22c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:54.892250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fc3bfca9-532f-4164-89f9-2e6724e22c7d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:54.893444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fc3bfca9-532f-4164-89f9-2e6724e22c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:54.961332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:54.976091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 512/1029] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:55.042922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea402e6-15b9-4428-a175-81086b8126f3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.043257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea402e6-15b9-4428-a175-81086b8126f3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.043477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea402e6-15b9-4428-a175-81086b8126f3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.044968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 518/1030] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:55.050571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.050925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.051194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.069407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:55.070576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:55.137765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 513/1031] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:55.251232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea402e6-15b9-4428-a175-81086b8126f3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:55.356706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea402e6-15b9-4428-a175-81086b8126f3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:55.391102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:55.391661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.391941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.392214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.393318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:55.395420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:55.397177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 519/1032] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:55.401893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:55.402618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:55.471434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:55.511569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72a7e3-5737-46ce-8344-76138b8d72d6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:55.512853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8aa2c6-9bfe-4cad-9e38-c841e1bdbd28 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:55.541658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:55.547187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:55.548014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.548296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.548559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.549214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:55.550691 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:55.552171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:55.552809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.553085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.553357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.564299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:55.570476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:55.571499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:55.572361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 514/1033] 10.222.0.72 () {56 vars in 1120 bytes} [Mon Sep 26 17:46:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:55.577764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:55.579187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 520/1034] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:46:55 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:46:55.699467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2c0d-8737-4581-a61c-3a0b8a527bc8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:55.739522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:55.740149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.740427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.740694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.741375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:55.741677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:55.743084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f204dc-7916-48df-b1b7-94e44b8006a0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:55.752050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 515/1035] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:46:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:55.758673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:55.818353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c767-6bc4-4491-90e2-2bb523d19412 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.818661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c767-6bc4-4491-90e2-2bb523d19412 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.819223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c767-6bc4-4491-90e2-2bb523d19412 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.821037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 516/1036] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:55.826280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.826566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.826862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.849548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:55.851223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:55.872505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f77a8d-0d2a-449d-b48f-692413ec31e9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:55.897819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:55.900330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:55.914529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:55.915192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:55.915474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:55.915749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:55.923309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 517/1037] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:55.929203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:55.930658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:55.946303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:55.947343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:55.953354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:55.953354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:55.953354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:55.953354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 521/1038] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:55.954705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.071319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7b74d-1d82-432c-9cf7-ae27cd054b4d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.106718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.107357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.107643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.107918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.108559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:56.108865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.110224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.126622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 518/1039] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:56.127820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.199424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.234527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48885481-87ab-4d9d-ac26-97cb98492db9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.244118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:56.271873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.272776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.273064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.273346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.281597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.283059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:46:56.295753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:56.296740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:56.298330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 522/1040] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:56.303534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.316221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6dc00-5c2a-45c7-bba2-016363eb13f6 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.338582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:56.356306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.356947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.357229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.357504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] PATH_INFO: `/v3/services/image` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.358089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.359488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:56.369306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:56.370360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:56.371280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 519/1041] 10.222.0.72 () {56 vars in 1105 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:56.376351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.413242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42301b9c-f06c-4966-bcce-7a9757da70e5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.448800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.449414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.449696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.449971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.450696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:56.451050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.452479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.476125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 523/1042] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:56.477191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.489928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bab5a-2730-4edc-97b6-d167b56e1a49 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.491552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:56.528844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.529459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.529794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.530017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.530665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] RBAC: Adding query filter params (name=image) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:56.531007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.532376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.537762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 520/1043] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:56.544011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.583388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:56.600499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874fce8-4e2b-4b1b-bf6f-6c72a851e609 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.636056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.636665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.636940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.637210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.661229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d1e7-2fac-440f-86b7-cdf76de03677 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.702322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.703215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.703737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.704288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.705246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] RBAC: Adding query filter params (type=image) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:56.705554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.707643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.708474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.709916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.718727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 521/1044] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:56.719523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.754313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:56.803239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:56.814946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 524/1045] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:46:56.820847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:56.840048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67f8bb-24a7-48e8-8c30-68c9bd0a7d04 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.876050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.876658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.876933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.877203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.877872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:56.879275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:56.902463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 522/1046] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:46:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:56.934819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bfd6c-bd3e-4988-9618-720100f5f514 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:56.985126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:56.985777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:56.986082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:56.986356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:56.998915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:57.000416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:57.019436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:57.020494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:46:57.026722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:57.026722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:46:57.026722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:57.026722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 525/1047] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:57.027026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:57.126309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:57.142953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45a58e-bf11-4768-b789-298997ff3751 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:57.177040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:57.177650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:57.177929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:57.178201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:57.178960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:57.179269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:57.180636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:57.192341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 523/1048] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:57.198096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:57.242380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:57.303315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28993a-fce9-4155-877b-4f5897c6fc8f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:57.337666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:57.338304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:57.338585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:57.338954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:57.345864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:57.347479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:46:57.359661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:57.360619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:46:57.361530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 526/1049] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:57.368933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:57.473364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e82dcc-63c4-4b6e-a745-7358e5bc65d2 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:57.508657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:57.509277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:57.509557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:57.509823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:57.510514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:57.510821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:57.512260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd3d0e-50c4-4699-9690-39ca06409783 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:57.529376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 524/1050] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:57.531113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:57.631159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:57.635129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6f77a-4df3-46e4-b55a-a9be40544408 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:57.669738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:57.670434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:57.670781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:57.671142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:57.671932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] RBAC: Adding query filter params (role.id=f0863cec30b84a909f64bc2f92c4a947, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1adfb9b41a1d4d31bccf5cc87b955526) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:57.672311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:57.673805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cb751-fe37-4376-a51a-d75df5195dfd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:57.686883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 527/1051] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:57 2022] GET /identity/v3/role_assignments?user.id=1adfb9b41a1d4d31bccf5cc87b955526&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=f0863cec30b84a909f64bc2f92c4a947 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:57.866009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944918-c508-4568-8b8d-62db42a86d5f None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:57.866436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944918-c508-4568-8b8d-62db42a86d5f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:57.866775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944918-c508-4568-8b8d-62db42a86d5f None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:57.868232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 525/1052] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:57.873734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:57.874212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:57.874655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:57.893695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:57.896103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:57.974871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 528/1053] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:58.249995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944918-c508-4568-8b8d-62db42a86d5f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:58.263900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:58.376890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944918-c508-4568-8b8d-62db42a86d5f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:58.413951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:58.414648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:58.415011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:58.415350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:58.426434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:58.431555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:58.432585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:46:58.433509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 526/1054] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:58.439041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:58.547707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:58.556041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75164ff9-b126-49aa-b48e-7102ce1548a4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:58.595675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:58.596464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:58.597526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:58.602867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:58.602867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:58.602867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:58.602867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:58.616041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 529/1055] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:58.617488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:58.704119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:58.740939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7883b37-9925-4ad5-9828-41d04a66b27b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:58.775958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:58.776592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:58.776873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:58.777152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] PATH_INFO: `/v3/users/368d02211bae406ea1bb2cc36527f9c3` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:58.790562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] RBAC: Authorizing `identity:get_user(user_id=368d02211bae406ea1bb2cc36527f9c3)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:58.792045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:58.798413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 527/1056] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:58 2022] GET /identity/v3/users/368d02211bae406ea1bb2cc36527f9c3 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:58.799683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:58.901671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea9397-c3a4-49df-991d-3ce1f956306e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:58.943397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98d02e-219b-4607-a11c-867d2cdb4902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:58.944267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98d02e-219b-4607-a11c-867d2cdb4902 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:58.944722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98d02e-219b-4607-a11c-867d2cdb4902 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:58.945190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98d02e-219b-4607-a11c-867d2cdb4902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:58.973985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:59.024125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98d02e-219b-4607-a11c-867d2cdb4902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:59.025698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98d02e-219b-4607-a11c-867d2cdb4902 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:59.132441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 530/1057] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:59.140582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:59.181046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7632f-eda6-4413-9955-6aa18b85dc1c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.181393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7632f-eda6-4413-9955-6aa18b85dc1c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.181607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7632f-eda6-4413-9955-6aa18b85dc1c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.183160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 531/1058] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:59.187783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.188070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.188350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.207437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:59.208627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:59.241437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d99b6-3d54-4493-9f21-b902e16ed206 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:59.260675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:59.268334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:59.280900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 532/1059] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:59.282980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:59.283651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.283938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.284222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.285401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] RBAC: Authorizing `identity:get_project(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:59.286858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:59.293582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 528/1060] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:59 2022] GET /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:59.294556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:59.396502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092aba1-d763-4271-8abf-b7edc7e7b7f5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:59.430124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:59.430734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.431046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.431325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.432049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530, user.id=368d02211bae406ea1bb2cc36527f9c3) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:59.432360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:59.433859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:59.447157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 533/1061] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:59 2022] GET /identity/v3/role_assignments?user.id=368d02211bae406ea1bb2cc36527f9c3&scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:59.553345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:59.666076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e3f2-1a83-49da-8b22-cf8d7e84916a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:59.701925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37bdcf-7d06-4933-9329-c685b57748b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:59.702539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37bdcf-7d06-4933-9329-c685b57748b0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.702910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37bdcf-7d06-4933-9329-c685b57748b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.703190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37bdcf-7d06-4933-9329-c685b57748b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.762791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:59.774343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37bdcf-7d06-4933-9329-c685b57748b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:59.776065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37bdcf-7d06-4933-9329-c685b57748b0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:59.887295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 529/1062] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:59.888369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:59.906705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a986f-a1fa-45d8-a8da-5f7037cd51a5 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.907138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a986f-a1fa-45d8-a8da-5f7037cd51a5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.907298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a986f-a1fa-45d8-a8da-5f7037cd51a5 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.908797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 530/1063] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:46:59.914313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:59.914604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:59.914903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:59.933452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:59.934569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:59.946460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:00.005036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1511a-d026-44ab-a38c-8eb2665608fc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:00.013250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 531/1064] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:47:00.045076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:00.045711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:00.045971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:00.046241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:00.057537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:00.059006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:47:00.073408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:00.074494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:47:00.075442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:47:00.075442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:00.075442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:00.075442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 534/1065] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:00.083583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:00.108702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:00.187984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f71c2-9020-4060-acd4-8783a5a4a2ad None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:00.223978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:00.224595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:00.224877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:00.225153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:00.225799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:00.226107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:00.227496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:00.239157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 532/1066] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:00.287712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:00.406359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2cad2-ce90-4fe4-9abf-dbdd3c9ff7c7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:00.407985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:00.446891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:00.447868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:00.448152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:00.448431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:00.460603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:00.465367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:00.466308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:00.467373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 535/1067] 10.222.0.72 () {56 vars in 1120 bytes} [Mon Sep 26 17:47:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:00.473445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:00.564437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:00.586098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adce2c-bef2-49af-bdf8-c4799ba4ada4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:00.620569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:00.621168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:00.621443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:00.621710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:00.622360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:00.622658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:00.624031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:00.633519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 533/1068] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:47:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:00.639059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:00.746390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5873fc1-f967-498e-bb20-ae94497faa0e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:00.781548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:00.782170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:00.782447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:00.782718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:00.793530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:00.794974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:47:00.809202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:00.810372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:47:00.811354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:47:00.811354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:00.811354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:00.811354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 536/1069] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:00.816909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:00.915270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92e263-a736-4a1d-a874-75ff148c0979 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:00.949152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:00.949746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:00.950021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:00.950288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:00.951009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:00.951310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:00.952650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:00.964955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 534/1070] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:00.970678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:00.989113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:01.074390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64414981-10ef-473b-a49b-442085f6f623 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:01.107851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:01.108473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.108747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.109016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.116301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:01.117710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:47:01.128531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:01.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:01.130675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 537/1071] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:47:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:01.135464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:01.139279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:01.235902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126655f-5ccc-422b-a2d1-70288f864882 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:01.268460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:01.269755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.270025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.270288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.271045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:01.271350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:01.273033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:01.290561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 535/1072] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:47:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:01.296347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:01.372505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:01.402279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224610c0-625f-4650-9aa1-e2261354c922 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:01.437011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:01.437637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.437917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.438191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.449579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:01.451083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:47:01.465907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:01.467087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:47:01.468008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:47:01.468008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:01.468008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:01.468008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 538/1073] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:47:01.472742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:01.533392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:01.571696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa925b-f332-47a4-a137-aecefc8b615e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:01.607954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:01.608583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.608867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.609144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.609812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:01.610121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:01.611576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ebe51-07f2-4e5d-9003-bc53093382f4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:01.624926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 536/1074] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:01.631146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:01.717221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03e291-c080-4069-8f31-239ec612552d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.717524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03e291-c080-4069-8f31-239ec612552d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.717784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03e291-c080-4069-8f31-239ec612552d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.719191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 537/1075] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:47:01.724190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.724481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.724762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.730864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1c29f-e69a-42a8-8847-2fc2b3d2ede4 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:01.743567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:01.744940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:01.757315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:01.775989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:01.776638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.777006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.777285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.784554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:01.786002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:47:01.797388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:01.798355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:01.799301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 539/1076] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:47:01.805244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:01.816352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 538/1077] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:47:01.904640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ad93c-1f56-4086-a59d-d5f67fbab2ed None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:01.937980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:01.938579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:01.938891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:01.939168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:01.941313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:01.941627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:01.943076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:01.955411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 540/1078] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:01.961602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:02.016533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:02.063576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6cfc4-57e4-4041-ae79-60896a81a907 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:02.096625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:02.098526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:02.099227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:02.099511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:02.099942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1adfb9b41a1d4d31bccf5cc87b955526/roles/f0863cec30b84a909f64bc2f92c4a947` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:02.101968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=1adfb9b41a1d4d31bccf5cc87b955526, role_id=f0863cec30b84a909f64bc2f92c4a947)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:02.103438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:02.116846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 539/1079] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:47:01 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/1adfb9b41a1d4d31bccf5cc87b955526/roles/f0863cec30b84a909f64bc2f92c4a947 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:47:02.152251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:02.210324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6b4ef-9a01-4b0d-8f9c-0baa2ef6539a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:02.246857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:02.247502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:02.247780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:02.248045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:02.259980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:02.264863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:02.266054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:02.267173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 541/1080] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:47:02.272845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:02.376814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464f611-7da0-42c1-a41f-456c9a57ee13 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:02.390147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:02.412178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925156-aace-435e-a183-05414b17f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:02.412773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925156-aace-435e-a183-05414b17f705 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:02.413055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925156-aace-435e-a183-05414b17f705 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:02.413326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925156-aace-435e-a183-05414b17f705 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:02.413978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a925156-aace-435e-a183-05414b17f705 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:02.414276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a925156-aace-435e-a183-05414b17f705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:02.415655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a925156-aace-435e-a183-05414b17f705 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:02.423533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 540/1081] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:02.429578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:02.528635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf85733-5e48-45e0-ad7e-2f04969f70e6 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:02.535828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:02.562825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:02.563472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:02.563753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:02.564026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] PATH_INFO: `/v3/users/368d02211bae406ea1bb2cc36527f9c3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:02.565128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] RBAC: Authorizing `identity:get_user(user_id=368d02211bae406ea1bb2cc36527f9c3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:02.567178 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:02.568769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 542/1082] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:02 2022] GET /identity/v3/users/368d02211bae406ea1bb2cc36527f9c3 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:02.574885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925156-aace-435e-a183-05414b17f705 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:02.675190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925156-aace-435e-a183-05414b17f705 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:02.708426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3553d-012c-4426-85f4-b3c5aa6ab1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:02.709046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3553d-012c-4426-85f4-b3c5aa6ab1cd None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:02.709321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3553d-012c-4426-85f4-b3c5aa6ab1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:02.709588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3553d-012c-4426-85f4-b3c5aa6ab1cd None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:02.710674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3553d-012c-4426-85f4-b3c5aa6ab1cd None admin] RBAC: Authorizing `identity:get_project(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:02.712107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3553d-012c-4426-85f4-b3c5aa6ab1cd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:02.713725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 541/1083] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:02 2022] GET /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:02.719737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:02.825372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b9395-726a-43ac-b1ea-52fb9b92208e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:02.870674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf277c-e5c4-447b-95b9-aa5b84f8f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:02.871364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf277c-e5c4-447b-95b9-aa5b84f8f8f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:02.871647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf277c-e5c4-447b-95b9-aa5b84f8f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:02.871922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf277c-e5c4-447b-95b9-aa5b84f8f8f9 None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530/users/368d02211bae406ea1bb2cc36527f9c3/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:02.873835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf277c-e5c4-447b-95b9-aa5b84f8f8f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530, user_id=368d02211bae406ea1bb2cc36527f9c3, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:02.875311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf277c-e5c4-447b-95b9-aa5b84f8f8f9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:02.889298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 543/1084] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:47:02 2022] PUT /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530/users/368d02211bae406ea1bb2cc36527f9c3/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:47:04.272016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c702e80-044e-4b9a-9e2e-71f1670c8526 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:04.272516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c702e80-044e-4b9a-9e2e-71f1670c8526 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:04.272516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c702e80-044e-4b9a-9e2e-71f1670c8526 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:04.273841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 542/1085] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:04.279132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:04.279552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:04.279673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:04.297744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:04.298945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:04.325022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:04.365737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 544/1086] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:04.632253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c702e80-044e-4b9a-9e2e-71f1670c8526 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:04.676552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:04.734862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c702e80-044e-4b9a-9e2e-71f1670c8526 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:04.768040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:04.769289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:04.769562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:04.769836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:04.780655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:04.785748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:04.786682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:47:04.787763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 543/1087] 10.222.0.72 () {56 vars in 1120 bytes} [Mon Sep 26 17:47:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:04.792882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:04.890582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ff05b-6466-47d4-ad83-7f41f39ff0a4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:04.923153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:04.923754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:04.924030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:04.924295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:04.925458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:04.925782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:04.927294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b39a7-77a4-4796-bb87-17dab73f660a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:04.936276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 545/1088] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:47:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:04.943377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.046037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dda51-9a1f-40d9-98af-9a2a7b62c52a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.047594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.056920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f50ee-af63-481c-975c-b4fad2981d03 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.057210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f50ee-af63-481c-975c-b4fad2981d03 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.057495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f50ee-af63-481c-975c-b4fad2981d03 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.058873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 546/1089] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:47:05.063746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.064030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.064312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.081595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.082412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.082696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.082999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.083111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:05.084218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:05.093792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.095244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.105097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:47:05.110033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:05.111105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:47:05.112011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:47:05.112011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:05.112011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:47:05.112011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 544/1090] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:05.117228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.152559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 547/1091] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:47:05.217253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa335232-25ff-4ccd-bd33-ccbd9a9465d3 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.249341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.249936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.250218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.250493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.251163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:05.251467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.252817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.265791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 545/1092] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:05.271840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.367902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbfc4a-878e-4aff-a7a8-ca7a973ceaa2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.402407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.403118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.403361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.403633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.411031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.412465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.422739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:47:05.424081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:05.426016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:05.427071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 548/1093] 10.222.0.72 () {56 vars in 1093 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:05.431274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.441868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.452400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.527113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b132b3-c5b9-4c06-8487-fea6b0f1efb1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.534523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a39ba-0bc8-4f31-b492-20e37ab90d88 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.560855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.561500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.561780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.562071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.571705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.572315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.572594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.572877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.573587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:05.573890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.574008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:05.575348 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:05.579068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:05.580018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:47:05.580936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 546/1094] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:47:05.586130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.597623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 549/1095] 10.222.0.72 () {56 vars in 1133 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:05.599508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.666941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.676141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.695098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec8c65-1cf3-42a5-939a-c30a3c907812 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.702313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c851b-62ac-492f-89c5-cad8cde71b90 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.728111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.728723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.729001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.729270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.729923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:05.730226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.731607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.735195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.735801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.736085 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.736360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.740110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 547/1096] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:05.745985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.804401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.805848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.847325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da30e7d-01f4-4c95-9f27-47f4e1c4fec9 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:05.878599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.886544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:05.887204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:05.887483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:05.887755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] PATH_INFO: `/v3/users/368d02211bae406ea1bb2cc36527f9c3` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:05.888907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] RBAC: Authorizing `identity:get_user(user_id=368d02211bae406ea1bb2cc36527f9c3)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:05.890339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:05.892120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 548/1097] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/users/368d02211bae406ea1bb2cc36527f9c3 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:05.899226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:05.902738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:05.911793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 550/1098] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:47:05.917526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.002482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246a829-8a6f-49d2-a844-b66a8852b294 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.019860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf7fc-2931-43b2-a747-aee765e5d2fb None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.037507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.038114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.038396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.038673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.054463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.055112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.055393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.055665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.065440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.066873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:47:06.081369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:06.082636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:47:06.083792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:47:06.083792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:47:06.083792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:06.083792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 551/1099] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:47:06.087996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.109426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.110896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.114279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:06.118519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:06.191619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a79b-d36e-449a-9c41-0e06b909e824 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.214565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 549/1100] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:06.220434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.241196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.242249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.242764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.243301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.244104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:06.244424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.245804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.259082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 552/1101] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:06.264615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.334616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941f83-0c54-49b3-8935-100e580945c4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.336933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:06.353214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:06.368918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ef059-a238-4ebc-bc7f-6d8daddb8b77 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.370380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.371053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.371333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.371608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.372724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] RBAC: Authorizing `identity:get_project(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.374153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.375832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 550/1102] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:06 2022] GET /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:06.381827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.405984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.406608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.406917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.407199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.414537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.416023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:47:06.427449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:06.428378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 553/1103] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:06.434394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.482394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097ed04-705e-4376-adb5-233bf8579c8c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.516266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.516875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.517155 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.517434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.518154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530, user.id=368d02211bae406ea1bb2cc36527f9c3) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:06.518974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.520540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.526601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 551/1104] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:06 2022] GET /identity/v3/role_assignments?user.id=368d02211bae406ea1bb2cc36527f9c3&scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:47:06.532340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a12be8-a5d5-466d-af00-b4dc739ee34c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.568633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.569506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.569789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.570053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.570742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:06.571074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.572515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adf67b-117d-4a13-9a28-af459687b13a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.586905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 554/1105] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:06.593298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:06.634959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:06.704593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e95a-8a74-43d8-85be-e5314d322473 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:06.740869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:06.741404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:06.741687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:06.741961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:06.742687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] RBAC: Adding query filter params (role.id=f0863cec30b84a909f64bc2f92c4a947, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=1adfb9b41a1d4d31bccf5cc87b955526) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:06.743052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:06.744485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74710f7f-9f2b-4a25-9c4e-7677728f5d6d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:06.750620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 552/1106] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:06 2022] GET /identity/v3/role_assignments?user.id=1adfb9b41a1d4d31bccf5cc87b955526&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=f0863cec30b84a909f64bc2f92c4a947 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:08.755906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe169-99cc-487f-8d39-84b03b0d2d6f None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:08.756475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe169-99cc-487f-8d39-84b03b0d2d6f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:08.756475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fe169-99cc-487f-8d39-84b03b0d2d6f None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:08.757820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 555/1107] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:47:08.763071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:08.763542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:08.763722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:08.782044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:08.783266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:08.849147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 553/1108] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:09.119899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fe169-99cc-487f-8d39-84b03b0d2d6f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:09.129276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:09.220273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fe169-99cc-487f-8d39-84b03b0d2d6f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:09.252695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea177-e6bb-458d-ae1a-247a230c5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:09.253916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea177-e6bb-458d-ae1a-247a230c5f60 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:09.254193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea177-e6bb-458d-ae1a-247a230c5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:09.254459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea177-e6bb-458d-ae1a-247a230c5f60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:09.255574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea177-e6bb-458d-ae1a-247a230c5f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:09.257058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea177-e6bb-458d-ae1a-247a230c5f60 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:09.258606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 556/1109] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:09.264329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:09.293609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:09.370490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47934f41-1590-4ce6-883c-d5771431f28d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:09.403268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753debc-35bc-4851-84a7-a334fa4c03dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:09.403864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753debc-35bc-4851-84a7-a334fa4c03dd None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:09.404140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753debc-35bc-4851-84a7-a334fa4c03dd None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:09.404410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753debc-35bc-4851-84a7-a334fa4c03dd None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:09.405055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753debc-35bc-4851-84a7-a334fa4c03dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:09.406468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753debc-35bc-4851-84a7-a334fa4c03dd None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:09.427132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 554/1110] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:47:09 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:47:11.609514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6581616-4f2c-410f-a511-19f8f4156c95 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:11.610048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6581616-4f2c-410f-a511-19f8f4156c95 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:11.610048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6581616-4f2c-410f-a511-19f8f4156c95 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:11.611453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 557/1111] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:11.616773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:11.617214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:11.617318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:11.635134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:11.636314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:11.701532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 555/1112] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:11.972204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6581616-4f2c-410f-a511-19f8f4156c95 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:12.044816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:12.071224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6581616-4f2c-410f-a511-19f8f4156c95 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:12.103842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:12.104452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:12.104780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:12.104984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] PATH_INFO: `/v3/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:12.106323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] RBAC: Authorizing `identity:get_role(role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:12.107702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:12.109018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 558/1113] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:11 2022] GET /identity/v3/roles/238ba40fcefe4923bb27057b1074fbef => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:12.115625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:12.132708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:12.217336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba673a5-0ff8-4f89-92a0-bbff0233d08e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:12.251195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:12.251762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:12.252045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:12.252322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] PATH_INFO: `/v3/users/b7a45c7179944fe0bbaa198a77d6df97` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:12.265360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] RBAC: Authorizing `identity:get_user(user_id=b7a45c7179944fe0bbaa198a77d6df97)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:12.266789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:12.268534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 556/1114] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:12 2022] GET /identity/v3/users/b7a45c7179944fe0bbaa198a77d6df97 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:12.274463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:12.372973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b5d3-0447-46f0-b5cf-68b991a72334 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:12.406644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:12.407270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:12.407552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:12.407825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:12.419917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:12.474179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:12.475647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:12.569829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 559/1115] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:12.575700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:12.648227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:12.676304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9fefd-0885-4bfe-a47f-0145f641b270 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:12.710779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79ab9d-2362-4e54-8fef-2f0dc500e2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:12.711434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79ab9d-2362-4e54-8fef-2f0dc500e2ba None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:12.711714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79ab9d-2362-4e54-8fef-2f0dc500e2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:12.711986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79ab9d-2362-4e54-8fef-2f0dc500e2ba None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:12.713084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79ab9d-2362-4e54-8fef-2f0dc500e2ba None admin] RBAC: Authorizing `identity:get_project(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:12.714508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79ab9d-2362-4e54-8fef-2f0dc500e2ba None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:12.716172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 557/1116] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:12 2022] GET /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:12.722349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:12.765611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:12.822256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e0c1e-3b9c-4b43-ba08-c9088d0f610c None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:12.855451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:12.856044 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:12.856326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:12.856597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:12.857313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] RBAC: Adding query filter params (role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530, user.id=b7a45c7179944fe0bbaa198a77d6df97) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:12.857622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:12.859071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8b1c3-657b-4382-a658-f5846062e570 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:12.864526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 560/1117] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:12 2022] GET /identity/v3/role_assignments?user.id=b7a45c7179944fe0bbaa198a77d6df97&scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530&role.id=238ba40fcefe4923bb27057b1074fbef => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:14.966585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22703f-ead6-4e51-924c-c9c21ac9bf5a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:14.967165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22703f-ead6-4e51-924c-c9c21ac9bf5a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:14.967165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22703f-ead6-4e51-924c-c9c21ac9bf5a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:14.968514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 558/1118] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:14.974422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:14.974868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:14.974993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:14.993885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:14.995105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:15.060545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 561/1119] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:15.334964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22703f-ead6-4e51-924c-c9c21ac9bf5a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:15.437712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22703f-ead6-4e51-924c-c9c21ac9bf5a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:15.471042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:15.472262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:15.472583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:15.472820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] PATH_INFO: `/v3/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:15.474205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] RBAC: Authorizing `identity:get_role(role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:15.475597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:15.476913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 559/1120] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:15 2022] GET /identity/v3/roles/238ba40fcefe4923bb27057b1074fbef => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:15.482709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:15.516873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:15.580749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d7153-a851-4d8d-8239-4c63dce793c7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:15.613929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:15.614493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:15.614774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:15.615076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] PATH_INFO: `/v3/users/b7a45c7179944fe0bbaa198a77d6df97` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:15.616184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] RBAC: Authorizing `identity:get_user(user_id=b7a45c7179944fe0bbaa198a77d6df97)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:15.617621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:15.619423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 562/1121] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:15 2022] GET /identity/v3/users/b7a45c7179944fe0bbaa198a77d6df97 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:15.625062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:15.641075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:15.727145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f230726-0493-4234-b5be-472385ba3f31 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:15.760170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87964b-a507-48c0-80ca-5615258e797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:15.760790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87964b-a507-48c0-80ca-5615258e797b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:15.761080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87964b-a507-48c0-80ca-5615258e797b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:15.761352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87964b-a507-48c0-80ca-5615258e797b None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:15.762452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87964b-a507-48c0-80ca-5615258e797b None admin] RBAC: Authorizing `identity:get_project(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:15.763898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87964b-a507-48c0-80ca-5615258e797b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:15.765516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 560/1122] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:15 2022] GET /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:15.772890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:15.871723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0e556-2650-4928-9d18-f7f06979f25b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:15.873921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:15.905767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44e5a1-43f2-4add-9bd2-0a5bdce53986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:15.906410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e5a1-43f2-4add-9bd2-0a5bdce53986 None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:15.906692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e5a1-43f2-4add-9bd2-0a5bdce53986 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:15.906996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e5a1-43f2-4add-9bd2-0a5bdce53986 None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530/users/b7a45c7179944fe0bbaa198a77d6df97/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:15.909156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44e5a1-43f2-4add-9bd2-0a5bdce53986 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530, user_id=b7a45c7179944fe0bbaa198a77d6df97, role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:15.910686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44e5a1-43f2-4add-9bd2-0a5bdce53986 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:15.922669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 563/1123] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:47:15 2022] PUT /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530/users/b7a45c7179944fe0bbaa198a77d6df97/roles/238ba40fcefe4923bb27057b1074fbef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:47:18.055354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497380-6485-4aa5-8f1d-d00cab7b220e None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:18.055860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497380-6485-4aa5-8f1d-d00cab7b220e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:18.055980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497380-6485-4aa5-8f1d-d00cab7b220e None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:18.057244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 561/1124] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:18.062430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:18.062829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:18.063009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:18.081268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:18.082450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:18.147386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 564/1125] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:18.419184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497380-6485-4aa5-8f1d-d00cab7b220e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:18.509127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:18.537913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497380-6485-4aa5-8f1d-d00cab7b220e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:18.571777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:18.572400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:18.572684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:18.572959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] PATH_INFO: `/v3/roles/238ba40fcefe4923bb27057b1074fbef` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:18.574331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] RBAC: Authorizing `identity:get_role(role_id=238ba40fcefe4923bb27057b1074fbef)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:18.575718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:18.577041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 562/1126] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:18 2022] GET /identity/v3/roles/238ba40fcefe4923bb27057b1074fbef => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:18.583204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:18.673083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:18.681330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c220-08f3-4e7c-abb8-f6a3ba766404 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:18.715641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:18.716265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:18.716548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:18.716822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] PATH_INFO: `/v3/users/b7a45c7179944fe0bbaa198a77d6df97` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:18.717915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=b7a45c7179944fe0bbaa198a77d6df97)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:18.719379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:18.720960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 565/1127] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:18 2022] GET /identity/v3/users/b7a45c7179944fe0bbaa198a77d6df97 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:18.727764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:18.835908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de7bb-b38d-4e03-8007-1f5fda42fb87 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:18.872389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:18.872984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:18.873259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:18.873526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:18.890314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:18.942519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:18.944150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:19.040756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 563/1128] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:19.046694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:19.146601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3256a-e244-4745-8efe-757206fff8b5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:19.183449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3620493-cb71-4d08-ba1d-1eabe5fdf20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:19.184504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620493-cb71-4d08-ba1d-1eabe5fdf20b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:19.184988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620493-cb71-4d08-ba1d-1eabe5fdf20b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:19.185459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620493-cb71-4d08-ba1d-1eabe5fdf20b None admin] PATH_INFO: `/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:19.187297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3620493-cb71-4d08-ba1d-1eabe5fdf20b None admin] RBAC: Authorizing `identity:get_project(project_id=1f7e40af2d9a4d6a8ae13ee6e3a4d530)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:19.189140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3620493-cb71-4d08-ba1d-1eabe5fdf20b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:19.191318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 566/1129] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:19 2022] GET /identity/v3/projects/1f7e40af2d9a4d6a8ae13ee6e3a4d530 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:19.197862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:19.244043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:19.295709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea0229-435b-491a-8e98-68f18b95d0b5 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:19.329370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:19.329963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:19.330242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:19.330511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:19.331333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] RBAC: Adding query filter params (role.id=238ba40fcefe4923bb27057b1074fbef, scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530, user.id=b7a45c7179944fe0bbaa198a77d6df97) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:19.331583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:19.332996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fd3ed-4dd8-4360-92b2-f81d1086139d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:19.338679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 564/1130] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:19 2022] GET /identity/v3/role_assignments?user.id=b7a45c7179944fe0bbaa198a77d6df97&scope.project.id=1f7e40af2d9a4d6a8ae13ee6e3a4d530&role.id=238ba40fcefe4923bb27057b1074fbef => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:21.494909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca8369-0c3d-4e27-b633-ab11483b5519 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:21.495448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca8369-0c3d-4e27-b633-ab11483b5519 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:21.495550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca8369-0c3d-4e27-b633-ab11483b5519 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:21.496834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 567/1131] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:21.502055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:21.502473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:21.502584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:21.525066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:21.526855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:21.598010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 565/1132] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:21.865765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca8369-0c3d-4e27-b633-ab11483b5519 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:21.901534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:21.975507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca8369-0c3d-4e27-b633-ab11483b5519 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:22.009609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a272e-7b0d-4a04-9f7d-38a50e885ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:22.010224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a272e-7b0d-4a04-9f7d-38a50e885ebd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:22.010499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a272e-7b0d-4a04-9f7d-38a50e885ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:22.010766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a272e-7b0d-4a04-9f7d-38a50e885ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:22.011872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a272e-7b0d-4a04-9f7d-38a50e885ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:22.013284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a272e-7b0d-4a04-9f7d-38a50e885ebd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:22.014885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 568/1133] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:22.021457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:22.061870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:22.119284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a0bce-b251-4108-9f5f-f9f0324832eb None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:22.152308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68495b-9a2b-476b-8587-68b5d132e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:22.152900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68495b-9a2b-476b-8587-68b5d132e728 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:22.153176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68495b-9a2b-476b-8587-68b5d132e728 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:22.153445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68495b-9a2b-476b-8587-68b5d132e728 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:22.154076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68495b-9a2b-476b-8587-68b5d132e728 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:22.155512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68495b-9a2b-476b-8587-68b5d132e728 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:22.182528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 566/1134] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:47:22 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:47:24.372745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afe674-6c21-4e48-942a-fb499a3aefaa None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:24.373262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afe674-6c21-4e48-942a-fb499a3aefaa None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:24.373262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afe674-6c21-4e48-942a-fb499a3aefaa None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:24.374622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 569/1135] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:24.380707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:24.381122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:24.381266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:24.399071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:24.400269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:24.466236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 567/1136] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:47:24.739182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afe674-6c21-4e48-942a-fb499a3aefaa None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:24.834970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:24.840519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afe674-6c21-4e48-942a-fb499a3aefaa None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:24.872346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fffce2-74b1-4e6f-a70e-f051786d2750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:24.872953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fffce2-74b1-4e6f-a70e-f051786d2750 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:24.873293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fffce2-74b1-4e6f-a70e-f051786d2750 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:24.873565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fffce2-74b1-4e6f-a70e-f051786d2750 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:24.874644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fffce2-74b1-4e6f-a70e-f051786d2750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:24.876094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fffce2-74b1-4e6f-a70e-f051786d2750 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:24.878377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 570/1137] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:24.884103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:24.907760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:24.991173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2debe8-d52b-4ba7-98df-2fd0230913c4 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:25.023959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb983f5-4bfd-4a4e-abde-b25beb404312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:25.024561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb983f5-4bfd-4a4e-abde-b25beb404312 None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:25.024845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb983f5-4bfd-4a4e-abde-b25beb404312 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:25.025123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb983f5-4bfd-4a4e-abde-b25beb404312 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:25.025770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb983f5-4bfd-4a4e-abde-b25beb404312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:25.027215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb983f5-4bfd-4a4e-abde-b25beb404312 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:25.048588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 568/1138] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:47:24 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:47:27.233433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103e0d2-19fc-49fe-ac22-19ee6b354f73 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:27.233961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103e0d2-19fc-49fe-ac22-19ee6b354f73 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:27.233961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103e0d2-19fc-49fe-ac22-19ee6b354f73 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:27.235290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 571/1139] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:27.241246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:27.241669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:27.241795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:27.259516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:27.260720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:27.325724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 569/1140] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:47:27.607547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103e0d2-19fc-49fe-ac22-19ee6b354f73 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:27.706513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103e0d2-19fc-49fe-ac22-19ee6b354f73 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:27.738996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:27.739649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:27.739977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:27.740260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:27.750148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:27.755536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:27.756657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:27.759940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 572/1141] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:27.762914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:27.773747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:27.860755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c8196-b3af-4c4b-83b2-2b9417be7bf4 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:27.894401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:27.895022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:27.895299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:27.895566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:27.896220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:27.896524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:27.897876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:27.905618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 570/1142] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:27.911664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:27.939027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:28.012063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032fb1-b85d-4dc0-87fc-8319444ad59b None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:28.044981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:28.045596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:28.045875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:28.046148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] PATH_INFO: `/v3/users/5e20a7ac81d645b68e9d614bdc3845f9` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:28.059525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] RBAC: Authorizing `identity:get_user(user_id=5e20a7ac81d645b68e9d614bdc3845f9)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:28.060978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:28.062583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 573/1143] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:27 2022] GET /identity/v3/users/5e20a7ac81d645b68e9d614bdc3845f9 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:28.068465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:28.136596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:28.171019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacf6b-23bc-460a-9a40-520ecf22b127 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:28.203299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:28.203896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:28.204178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:28.204452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:28.271033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:28.272470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:28.342105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:28.376410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 571/1144] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:28.383070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:28.465716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:28.482368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d0f1b-0316-4eb4-a94a-af74415b50d9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:28.516515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f64d6-7c1a-45df-826b-060a74025459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:28.517129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f64d6-7c1a-45df-826b-060a74025459 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:28.517410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f64d6-7c1a-45df-826b-060a74025459 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:28.517681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f64d6-7c1a-45df-826b-060a74025459 None admin] PATH_INFO: `/v3/projects/485f75a2355e410780e20dd426adc2e1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:28.518899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f64d6-7c1a-45df-826b-060a74025459 None admin] RBAC: Authorizing `identity:get_project(project_id=485f75a2355e410780e20dd426adc2e1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:28.520320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f64d6-7c1a-45df-826b-060a74025459 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:28.521943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 574/1145] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:28 2022] GET /identity/v3/projects/485f75a2355e410780e20dd426adc2e1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:28.528109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:28.645228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad207495-2081-4ca2-b935-8f99a7467382 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:28.685457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:28.686060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:28.686340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:28.686614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:28.687369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=485f75a2355e410780e20dd426adc2e1, user.id=5e20a7ac81d645b68e9d614bdc3845f9) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:28.687682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:28.689106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5173f9-07b2-44e2-a864-214fe8a2cc3d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:28.696022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 572/1146] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:28 2022] GET /identity/v3/role_assignments?user.id=5e20a7ac81d645b68e9d614bdc3845f9&scope.project.id=485f75a2355e410780e20dd426adc2e1&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:30.990970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664a231-3275-4b81-95b8-3ba9d62a7e35 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:30.991387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664a231-3275-4b81-95b8-3ba9d62a7e35 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:30.991527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664a231-3275-4b81-95b8-3ba9d62a7e35 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:30.992891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 575/1147] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:30.998111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:30.998510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:30.998656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:31.017469 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:31.019002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:31.045188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:31.085769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 573/1148] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:31.356005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664a231-3275-4b81-95b8-3ba9d62a7e35 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:31.455897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664a231-3275-4b81-95b8-3ba9d62a7e35 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:31.489365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:31.489983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:31.490267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:31.490544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:31.500499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:31.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:31.507114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:31.508031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 576/1149] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:31.513005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:31.609886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5477c6b-c59b-4a3b-86e1-ce096ff3dd18 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:31.642134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:31.642725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:31.643032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:31.644040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:31.644772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:31.645073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:31.646422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:31.654109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 574/1150] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:31.660199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:31.675413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:31.758523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b4f94-2284-49eb-9074-eb7b655c6541 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:31.791535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:31.792152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:31.792433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:31.792705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] PATH_INFO: `/v3/users/5e20a7ac81d645b68e9d614bdc3845f9` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:31.793805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] RBAC: Authorizing `identity:get_user(user_id=5e20a7ac81d645b68e9d614bdc3845f9)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:31.795262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:31.796832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 577/1151] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:31 2022] GET /identity/v3/users/5e20a7ac81d645b68e9d614bdc3845f9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:31.803483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:31.831653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:31.904050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd2986-f80c-42b2-8529-30a8435944b7 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:31.936587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c35fa1-ef83-42db-8499-3f0de1a59d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:31.937204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35fa1-ef83-42db-8499-3f0de1a59d5d None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:31.937485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35fa1-ef83-42db-8499-3f0de1a59d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:31.937759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c35fa1-ef83-42db-8499-3f0de1a59d5d None admin] PATH_INFO: `/v3/projects/485f75a2355e410780e20dd426adc2e1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:31.938893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c35fa1-ef83-42db-8499-3f0de1a59d5d None admin] RBAC: Authorizing `identity:get_project(project_id=485f75a2355e410780e20dd426adc2e1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:31.940922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c35fa1-ef83-42db-8499-3f0de1a59d5d None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:31.942578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 575/1152] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:31 2022] GET /identity/v3/projects/485f75a2355e410780e20dd426adc2e1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:31.948879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:32.027628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:32.046373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f602-4df9-4613-b965-942ca2fd41fc None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:32.080076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ec0d7-fd6a-4acf-99e1-a7786418263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:32.080890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec0d7-fd6a-4acf-99e1-a7786418263a None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:32.081298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec0d7-fd6a-4acf-99e1-a7786418263a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:32.081580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec0d7-fd6a-4acf-99e1-a7786418263a None admin] PATH_INFO: `/v3/projects/485f75a2355e410780e20dd426adc2e1/users/5e20a7ac81d645b68e9d614bdc3845f9/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:32.083460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ec0d7-fd6a-4acf-99e1-a7786418263a None admin] RBAC: Authorizing `identity:create_grant(project_id=485f75a2355e410780e20dd426adc2e1, user_id=5e20a7ac81d645b68e9d614bdc3845f9, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:32.084878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ec0d7-fd6a-4acf-99e1-a7786418263a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:32.097342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 578/1153] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:47:31 2022] PUT /identity/v3/projects/485f75a2355e410780e20dd426adc2e1/users/5e20a7ac81d645b68e9d614bdc3845f9/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:47:34.256111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2308-8a5b-4573-80a2-496b9605908a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:34.256639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2308-8a5b-4573-80a2-496b9605908a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:34.256639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e2308-8a5b-4573-80a2-496b9605908a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:34.257955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 576/1154] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:34.263293 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:34.263698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:34.263844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:34.282113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:34.283342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:34.349989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 579/1155] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:34.638748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e2308-8a5b-4573-80a2-496b9605908a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:34.731304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:34.739539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e2308-8a5b-4573-80a2-496b9605908a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:34.773303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:34.773958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:34.774243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:34.774519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:34.786367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:34.791271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:34.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:47:34.793272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 577/1156] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:34.798450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:34.866382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:34.897431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd4a7a-033d-4916-b170-b3997604dfc1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:34.931680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:34.932282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:34.932559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:34.932830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:34.933485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:34.933789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:34.935176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:34.943672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 580/1157] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:34.950115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:35.050909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f381bd-f61a-43f2-80ba-6317b6dcae02 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:35.084012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:35.084657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:35.084939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:35.085215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] PATH_INFO: `/v3/users/5e20a7ac81d645b68e9d614bdc3845f9` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:35.086323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] RBAC: Authorizing `identity:get_user(user_id=5e20a7ac81d645b68e9d614bdc3845f9)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:35.087816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:35.089540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 578/1158] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:34 2022] GET /identity/v3/users/5e20a7ac81d645b68e9d614bdc3845f9 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:35.095374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:35.192528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937e2cc-dbda-4640-a72b-96d75c64668a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:35.220887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:35.226016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:35.226611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:35.226920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:35.227198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:35.291273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:35.292705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:35.387270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 581/1159] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:35.394438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:35.418260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:35.493029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a218f9-2049-4a14-911e-3e688f18211b None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:35.526490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474beaf-4641-4656-84ac-152f3fe1c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:35.527143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474beaf-4641-4656-84ac-152f3fe1c567 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:35.527428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474beaf-4641-4656-84ac-152f3fe1c567 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:35.527706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474beaf-4641-4656-84ac-152f3fe1c567 None admin] PATH_INFO: `/v3/projects/485f75a2355e410780e20dd426adc2e1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:35.528809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474beaf-4641-4656-84ac-152f3fe1c567 None admin] RBAC: Authorizing `identity:get_project(project_id=485f75a2355e410780e20dd426adc2e1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:35.530232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474beaf-4641-4656-84ac-152f3fe1c567 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:35.531888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 579/1160] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:35 2022] GET /identity/v3/projects/485f75a2355e410780e20dd426adc2e1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:35.538134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:35.569419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:35.637765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813ed0-ccf1-499f-abc1-ed2dd11366aa None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:35.672550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:35.673146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:35.673427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:35.673707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:35.674423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=485f75a2355e410780e20dd426adc2e1, user.id=5e20a7ac81d645b68e9d614bdc3845f9) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:35.674726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:35.676274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d30fa-51c1-4a2f-941f-538d07c34fd0 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:35.681911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 582/1161] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:35 2022] GET /identity/v3/role_assignments?user.id=5e20a7ac81d645b68e9d614bdc3845f9&scope.project.id=485f75a2355e410780e20dd426adc2e1&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:37.869229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835229-208f-49cb-92bf-7f08fe4c3d4a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:37.869769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835229-208f-49cb-92bf-7f08fe4c3d4a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:37.869769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe835229-208f-49cb-92bf-7f08fe4c3d4a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:37.871153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 580/1162] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:37.876433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:37.876852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:37.876991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:37.895573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:37.896758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:37.962985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 583/1163] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:38.272798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe835229-208f-49cb-92bf-7f08fe4c3d4a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:38.362255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:38.372881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe835229-208f-49cb-92bf-7f08fe4c3d4a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:38.405615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:38.406209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:38.406486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:38.406755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:38.470431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:38.472687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:38.563574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:38.574767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 581/1164] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:38.580676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:38.607352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:38.678499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fff37-4f21-4354-a947-ac6164425fa8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:38.712646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:38.713263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:38.713545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:38.713821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:38.720387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:38.722634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-19c15744-9ac4-4ada-b1f7-64d602baf072 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:38.734579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:38.735809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:38.736706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 584/1165] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:47:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:38.742004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:38.838706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b4d06-63f6-44cf-bfa1-8f1a227dd55e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:38.871698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:38.872280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:38.872560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:38.872831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:38.873462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:38.873763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:38.875133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af785cf4-3f8e-414c-8c21-9be836ce9ba2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:38.882175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 582/1166] 10.222.0.72 () {56 vars in 1115 bytes} [Mon Sep 26 17:47:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:41.368012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f855d-5978-4719-9b6d-64bbaf733086 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:41.368557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f855d-5978-4719-9b6d-64bbaf733086 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:41.368557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f855d-5978-4719-9b6d-64bbaf733086 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:41.369856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 585/1167] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:41.375358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d3a1-34b4-48bc-9c40-d51f707c9989 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:41.375787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d3a1-34b4-48bc-9c40-d51f707c9989 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:41.375899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d3a1-34b4-48bc-9c40-d51f707c9989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:41.395197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8bc6d3a1-34b4-48bc-9c40-d51f707c9989 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:41.396393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8bc6d3a1-34b4-48bc-9c40-d51f707c9989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:41.445656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:41.463462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 583/1168] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:41.740970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f855d-5978-4719-9b6d-64bbaf733086 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:41.825566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:41.841138 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f855d-5978-4719-9b6d-64bbaf733086 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:41.873781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5227900-35a6-4d3d-a937-8d6a39af7296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:41.874378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227900-35a6-4d3d-a937-8d6a39af7296 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:41.874662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227900-35a6-4d3d-a937-8d6a39af7296 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:41.874964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227900-35a6-4d3d-a937-8d6a39af7296 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:41.875635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227900-35a6-4d3d-a937-8d6a39af7296 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:41.877022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227900-35a6-4d3d-a937-8d6a39af7296 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:41.893329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 586/1169] 10.222.0.72 () {60 vars in 1138 bytes} [Mon Sep 26 17:47:41 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:47:44.064103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4fbc6-02a5-4586-a26c-e9275c0a4e20 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:44.064635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4fbc6-02a5-4586-a26c-e9275c0a4e20 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:44.064635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4fbc6-02a5-4586-a26c-e9275c0a4e20 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:44.066678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 584/1170] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:44.072206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:44.072586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:44.072757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:44.090708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:44.091955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:44.176728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 587/1171] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:44.450967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4fbc6-02a5-4586-a26c-e9275c0a4e20 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:44.555867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4fbc6-02a5-4586-a26c-e9275c0a4e20 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:44.590062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28b260-142c-45ed-8cd3-e552414a26ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:44.590682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b260-142c-45ed-8cd3-e552414a26ce None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:44.590989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b260-142c-45ed-8cd3-e552414a26ce None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:44.591260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28b260-142c-45ed-8cd3-e552414a26ce None admin] PATH_INFO: `/v3/domains/192bd74952c34b49beb23594933eb623` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:44.601962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28b260-142c-45ed-8cd3-e552414a26ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=192bd74952c34b49beb23594933eb623)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:44.603415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28b260-142c-45ed-8cd3-e552414a26ce None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:44.604996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 585/1172] 10.222.0.72 () {56 vars in 1183 bytes} [Mon Sep 26 17:47:44 2022] GET /identity/v3/domains/192bd74952c34b49beb23594933eb623 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:44.611047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:44.685562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:44.709490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81164583-55aa-4981-96f4-111edf8d6ac1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:44.742704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc911c1-4290-4b24-8e5a-9e7f261eacaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:44.743337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc911c1-4290-4b24-8e5a-9e7f261eacaf None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:44.743620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc911c1-4290-4b24-8e5a-9e7f261eacaf None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:44.743892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc911c1-4290-4b24-8e5a-9e7f261eacaf None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:44.744538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc911c1-4290-4b24-8e5a-9e7f261eacaf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:44.745969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc911c1-4290-4b24-8e5a-9e7f261eacaf None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:44.773102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 588/1173] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:47:44 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:47:46.960799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a65ca8-58b0-4a8f-9480-7a1b582743d4 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:46.961370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a65ca8-58b0-4a8f-9480-7a1b582743d4 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:46.961370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a65ca8-58b0-4a8f-9480-7a1b582743d4 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:46.962684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 586/1174] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:46.968037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:46.968443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:46.968592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:46.986560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:46.988415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:47.053676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 589/1175] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:47.331698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a65ca8-58b0-4a8f-9480-7a1b582743d4 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:47.378533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:47.433421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a65ca8-58b0-4a8f-9480-7a1b582743d4 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:47.465468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fee8a-53e1-4bfb-8983-800eacb2725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:47.466085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fee8a-53e1-4bfb-8983-800eacb2725e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:47.466365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fee8a-53e1-4bfb-8983-800eacb2725e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:47.466638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fee8a-53e1-4bfb-8983-800eacb2725e None admin] PATH_INFO: `/v3/domains/192bd74952c34b49beb23594933eb623` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:47.467753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fee8a-53e1-4bfb-8983-800eacb2725e None admin] RBAC: Authorizing `identity:get_domain(domain_id=192bd74952c34b49beb23594933eb623)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:47.469173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fee8a-53e1-4bfb-8983-800eacb2725e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:47.470739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 587/1176] 10.222.0.72 () {56 vars in 1183 bytes} [Mon Sep 26 17:47:47 2022] GET /identity/v3/domains/192bd74952c34b49beb23594933eb623 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:47.476932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:47.533361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:47.575820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb437f-ad82-4132-946d-7e6acf222a45 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:47.611152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12530c-b701-4191-b192-fbb029dd9f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:47.611688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12530c-b701-4191-b192-fbb029dd9f86 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:47.612080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12530c-b701-4191-b192-fbb029dd9f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:47.612361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12530c-b701-4191-b192-fbb029dd9f86 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:47.613023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12530c-b701-4191-b192-fbb029dd9f86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:47.614453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12530c-b701-4191-b192-fbb029dd9f86 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:47.635362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 590/1177] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:47:47 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:47:49.901607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdeb7-0bc8-41db-afd8-106bf302120b None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:49.902164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdeb7-0bc8-41db-afd8-106bf302120b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:49.902164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdeb7-0bc8-41db-afd8-106bf302120b None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:49.903528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 588/1178] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:49.909511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:49.909963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:49.910116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:49.928213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:49.929379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:49.996812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 591/1179] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:50.279496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdeb7-0bc8-41db-afd8-106bf302120b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:50.399716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdeb7-0bc8-41db-afd8-106bf302120b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:50.412503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:50.433454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:50.434076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:50.434360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:50.434636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:50.445958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:50.450752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:50.451891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:47:50.452938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 589/1180] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:50.458903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:50.501726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:50.554845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b65ed-4ee9-4d79-b7f6-e66aa77a9b3a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:50.589725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:50.590434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:50.590717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:50.591109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:50.591763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:50.592070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:50.593466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:50.601688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 592/1181] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:50.607603 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:50.707279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd3ef-66f3-4235-b6bf-98364e18b337 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:50.739184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:50.739794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:50.740071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:50.740338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] PATH_INFO: `/v3/users/282d33cb6cb24f629e8425995def11e1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:50.752844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=282d33cb6cb24f629e8425995def11e1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:50.754424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:50.756021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 590/1182] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:50 2022] GET /identity/v3/users/282d33cb6cb24f629e8425995def11e1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:50.762001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:50.861397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3c7c2-62df-42d6-b35c-53e8790ea7b7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:50.868286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:50.895254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:50.895850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:50.896129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:50.896397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:50.962522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:50.964002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:51.060750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 593/1183] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:51.066716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:51.097148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:51.164446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc36830-6762-4e51-a166-d17bf70a7ce8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:51.197661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f927b4a-bb69-479c-920b-f3398e929c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:51.198305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927b4a-bb69-479c-920b-f3398e929c8b None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:51.198586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927b4a-bb69-479c-920b-f3398e929c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:51.198881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f927b4a-bb69-479c-920b-f3398e929c8b None admin] PATH_INFO: `/v3/projects/997f47eab67640759bb222e8e3587dc7` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:51.199984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f927b4a-bb69-479c-920b-f3398e929c8b None admin] RBAC: Authorizing `identity:get_project(project_id=997f47eab67640759bb222e8e3587dc7)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:51.201400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f927b4a-bb69-479c-920b-f3398e929c8b None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:51.203036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 591/1184] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:51 2022] GET /identity/v3/projects/997f47eab67640759bb222e8e3587dc7 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:51.209260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:51.217124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:51.306932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccae243-b5ff-4ad4-b814-a94f34976201 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:51.339841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:51.340432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:51.340713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:51.340985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:51.341693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=997f47eab67640759bb222e8e3587dc7, user.id=282d33cb6cb24f629e8425995def11e1) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:51.342001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:51.343452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71468-f27b-4da1-b816-af27471aa9f6 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:51.348798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 594/1185] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:51 2022] GET /identity/v3/role_assignments?user.id=282d33cb6cb24f629e8425995def11e1&scope.project.id=997f47eab67640759bb222e8e3587dc7&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:53.504553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8790b-dcb9-499c-8d56-75317e1d3ded None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:53.505115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8790b-dcb9-499c-8d56-75317e1d3ded None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:53.505115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8790b-dcb9-499c-8d56-75317e1d3ded None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:53.506460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 592/1186] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:53.511565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:53.511972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:53.512121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:53.530803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:53.532039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:53.593703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:53.598944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 595/1187] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:53.870361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8790b-dcb9-499c-8d56-75317e1d3ded None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:53.967843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:53.973738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8790b-dcb9-499c-8d56-75317e1d3ded None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:54.007106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:54.007719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:54.007995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:54.008263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:54.018130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:54.023109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:54.024299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:47:54.025533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 593/1188] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:54.030285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:54.130180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1c50-69d5-4bd0-966b-0ff57263df31 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:54.162994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:54.163581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:54.163858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:54.164126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:54.164776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:54.165076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:54.166426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:54.174061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 596/1189] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:54.180613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:54.278769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f509b-d41f-4fb2-a76a-f1d732861b2f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:54.311673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:54.312280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:54.312559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:54.312831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] PATH_INFO: `/v3/users/282d33cb6cb24f629e8425995def11e1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:54.313926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] RBAC: Authorizing `identity:get_user(user_id=282d33cb6cb24f629e8425995def11e1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:54.316115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:54.317671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 594/1190] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:54 2022] GET /identity/v3/users/282d33cb6cb24f629e8425995def11e1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:54.323703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:54.382667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:54.420129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1673d-2afe-4739-9cc1-769ffb39bd98 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:54.453549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26139eb7-d5fc-4c34-a23c-f5eda389529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:54.454165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26139eb7-d5fc-4c34-a23c-f5eda389529c None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:54.454443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26139eb7-d5fc-4c34-a23c-f5eda389529c None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:54.454716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26139eb7-d5fc-4c34-a23c-f5eda389529c None admin] PATH_INFO: `/v3/projects/997f47eab67640759bb222e8e3587dc7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:54.455958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26139eb7-d5fc-4c34-a23c-f5eda389529c None admin] RBAC: Authorizing `identity:get_project(project_id=997f47eab67640759bb222e8e3587dc7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:54.457369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26139eb7-d5fc-4c34-a23c-f5eda389529c None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:54.459008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 597/1191] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:54 2022] GET /identity/v3/projects/997f47eab67640759bb222e8e3587dc7 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:54.464975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:54.492126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:54.562439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f46ae-57b3-4116-930d-6b2efa9aae8a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:54.595844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e388e5e-231d-4948-808c-4bba14dc6c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:54.596616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e388e5e-231d-4948-808c-4bba14dc6c73 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:54.596893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e388e5e-231d-4948-808c-4bba14dc6c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:54.597170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e388e5e-231d-4948-808c-4bba14dc6c73 None admin] PATH_INFO: `/v3/projects/997f47eab67640759bb222e8e3587dc7/users/282d33cb6cb24f629e8425995def11e1/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:54.599075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e388e5e-231d-4948-808c-4bba14dc6c73 None admin] RBAC: Authorizing `identity:create_grant(project_id=997f47eab67640759bb222e8e3587dc7, user_id=282d33cb6cb24f629e8425995def11e1, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:54.600490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e388e5e-231d-4948-808c-4bba14dc6c73 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:54.613492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 595/1192] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:47:54 2022] PUT /identity/v3/projects/997f47eab67640759bb222e8e3587dc7/users/282d33cb6cb24f629e8425995def11e1/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:47:56.775265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1502b7c-3d6c-4670-978e-574a4bb96283 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:56.775825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1502b7c-3d6c-4670-978e-574a4bb96283 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:56.775825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1502b7c-3d6c-4670-978e-574a4bb96283 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:56.777169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 598/1193] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:56.782385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:56.782817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:56.782990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:56.801200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:56.802377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:56.866752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 596/1194] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:47:57.134618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1502b7c-3d6c-4670-978e-574a4bb96283 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:57.235775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1502b7c-3d6c-4670-978e-574a4bb96283 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:57.263320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:57.269479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:57.270097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:57.270377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:57.270651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:57.281065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:47:57.286364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:47:57.287531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:47:57.288421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 599/1195] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:47:57.293699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:57.311313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:57.391088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a04b2-e52a-4eae-bd67-70608a3776a6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:57.424367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:57.425069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:57.425344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:57.425616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:57.426269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:57.426571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:57.427952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:57.435732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 597/1196] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:57.441820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:57.540214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e42e6c-b3bf-40ed-b45f-7fd2de892f31 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:57.572457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:57.573070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:57.573345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:57.573613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] PATH_INFO: `/v3/users/282d33cb6cb24f629e8425995def11e1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:57.574694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] RBAC: Authorizing `identity:get_user(user_id=282d33cb6cb24f629e8425995def11e1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:57.576144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:57.577709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 600/1197] 10.222.0.72 () {56 vars in 1177 bytes} [Mon Sep 26 17:47:57 2022] GET /identity/v3/users/282d33cb6cb24f629e8425995def11e1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:57.584986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:57.663684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:57.684875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d9449-0858-4ab6-b51f-fe80b3b0909a None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:57.718535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:57.719164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:57.719447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:57.719721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:57.784126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:57.785546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:57.858240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:57.879890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 598/1198] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:47:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:47:57.885813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:57.943128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:57.985381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b8428-3135-4a1a-a2c2-c6b44bd413af None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:58.018657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7113aa1-6215-4418-a284-78ea50cf9b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:58.019307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7113aa1-6215-4418-a284-78ea50cf9b41 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:58.019593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7113aa1-6215-4418-a284-78ea50cf9b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:58.019869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7113aa1-6215-4418-a284-78ea50cf9b41 None admin] PATH_INFO: `/v3/projects/997f47eab67640759bb222e8e3587dc7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:58.020978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7113aa1-6215-4418-a284-78ea50cf9b41 None admin] RBAC: Authorizing `identity:get_project(project_id=997f47eab67640759bb222e8e3587dc7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:58.022409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7113aa1-6215-4418-a284-78ea50cf9b41 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:58.024081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 601/1199] 10.222.0.72 () {56 vars in 1186 bytes} [Mon Sep 26 17:47:57 2022] GET /identity/v3/projects/997f47eab67640759bb222e8e3587dc7 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:47:58.030249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:47:58.127631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ffe8b-86af-41f1-adc7-ffb9d3251825 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:47:58.161057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:47:58.161647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:58.161926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:58.162199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:58.162944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=997f47eab67640759bb222e8e3587dc7, user.id=282d33cb6cb24f629e8425995def11e1) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:47:58.163256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:47:58.164658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c727d-900a-4984-865c-9f0f461755b6 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:47:58.170246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 599/1200] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:47:58 2022] GET /identity/v3/role_assignments?user.id=282d33cb6cb24f629e8425995def11e1&scope.project.id=997f47eab67640759bb222e8e3587dc7&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:08.014490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=37 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:08.167703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=79 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:54.597337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82499b1d-8f7b-4743-b051-2ed2581adf03 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:54.598091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82499b1d-8f7b-4743-b051-2ed2581adf03 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:54.598091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82499b1d-8f7b-4743-b051-2ed2581adf03 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:54.599761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 602/1201] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:54.610752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:54.611387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:54.611387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:54.636472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:54.637670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:54.728146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 600/1202] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:48:55.005837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82499b1d-8f7b-4743-b051-2ed2581adf03 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:55.134526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82499b1d-8f7b-4743-b051-2ed2581adf03 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:55.180702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:55.181429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:55.181749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:55.181999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:55.199150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:55.200604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:55.216865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:55.218945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:55.220102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:55.220102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:55.220102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:48:55.220102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 603/1203] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:55.225159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:55.409398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d17413-bb5e-4510-b3cc-ad02ac49f12e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:55.464231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:55.477339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:55.478105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:55.478522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:55.478819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:55.479589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:55.479950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:55.481455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5fc96-f71e-4b8e-8aa7-e4a8b787cae2 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:55.503030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 601/1204] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:55.508890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:55.582704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:55.643511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd516d-ea47-4be9-82e3-bdd5dc20c17a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:55.685520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50868a-fc8a-41c4-8f09-5f556677046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:55.686428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50868a-fc8a-41c4-8f09-5f556677046a None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:55.686703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50868a-fc8a-41c4-8f09-5f556677046a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:55.687020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50868a-fc8a-41c4-8f09-5f556677046a None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:55.687676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50868a-fc8a-41c4-8f09-5f556677046a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:55.689133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50868a-fc8a-41c4-8f09-5f556677046a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:55.719172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 604/1205] 10.222.0.72 () {60 vars in 1132 bytes} [Mon Sep 26 17:48:55 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:48:58.102356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399323-b5ef-4d84-95b6-570e7d705fbb None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:58.103121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399323-b5ef-4d84-95b6-570e7d705fbb None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:58.103121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399323-b5ef-4d84-95b6-570e7d705fbb None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:58.104589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 602/1206] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:58.111394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:58.112049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:58.112049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:58.132855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:58.134360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:58.219308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 605/1207] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:58.531769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9399323-b5ef-4d84-95b6-570e7d705fbb None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:58.679084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9399323-b5ef-4d84-95b6-570e7d705fbb None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:58.727152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:58.727785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:58.728068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:58.728342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:58.741673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:58.746504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:58.748064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:48:58.749042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 603/1208] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:48:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:58.753875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:58.804005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:58.883316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28465368-d1e8-41eb-af4a-28807afe5213 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:58.929709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:58.930496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:58.930892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:58.931266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:58.932065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:58.932486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:58.934017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:58.945079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 606/1209] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:48:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:48:58.955011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:59.026066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:59.108271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db3ea6-57b2-4732-9ba5-5aeffdf7cd04 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:59.152369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:59.155130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:59.155130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:59.155130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:59.169825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:59.171402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:59.191016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:59.192196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:59.193116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:59.193116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:59.193116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:48:59.193116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 604/1210] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:59.198648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:59.331066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:59.335299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544b5fb-e8c5-42d7-a713-799c71dedde3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:59.380978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:59.380978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:59.381256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:59.381339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:59.381922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:59.382230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:59.383694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:59.401240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 607/1211] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:59.402631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:59.528828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb338-ff40-499c-acc4-87ab39998cca None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:59.530582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:59.568796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:59.569434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:59.569969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:59.570247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:59.577926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:59.579407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:59.592667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:59.593834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:48:59.594993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 605/1212] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:59.602083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:59.726547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68597387-4c3d-4750-afbf-dd5cfd3ce149 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:59.766206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:59.766827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:59.767163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:59.768093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:59.768872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:59.769271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:59.771013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:59.797122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 608/1213] 10.222.0.72 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:59.798310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:59.920435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3484ac-9fbf-47bf-a5c8-b5e3690543c8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:59.960713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:59.961444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:59.961799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:59.962139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:00.003376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:00.046498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:00.048243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:00.162640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 606/1214] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:48:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:49:00.172152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:00.192804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:00.284031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706ef69-80ef-4996-8254-2ca99028d04a None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:00.333764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:00.334799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:00.335118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:00.335440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:00.347406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:00.349179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:00.367466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:00.369219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:00.370604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:00.370604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:00.370604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:00.370604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 609/1215] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:00.374499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:00.441969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:00.483193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ddd5-32cd-471f-99d1-714206ed5c86 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:00.526367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:00.527148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:00.527421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:00.527722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:00.528336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:00.528639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:00.530015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:00.544042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 607/1216] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:00.548266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:00.631640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:00.667753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dfce2-f70c-4bb0-8902-626e70cb07a3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:00.705092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:00.706153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:00.706435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:00.706703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:00.713790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:00.715548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:49:00.729157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:00.730398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:00.731647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 610/1217] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:00.736426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:00.846278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a196acb-96e5-48cb-b7ae-e2138beaa4ea None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:00.869900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:00.884233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:00.884878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:00.885165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:00.885523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:00.886125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:00.886432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:00.887897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc34c2-e636-4a27-b410-16d1465f41ab None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:00.900789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 608/1218] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:49:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:00.906950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:01.022456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5bd71-41f0-42d1-936e-2ab2f33ff9d1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:01.054584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:01.061416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:01.062027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:01.062309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:01.062657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:01.063395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=92937af60f80421094f4bdf6c7620283) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:01.063702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:01.065905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f4d0-456e-44d0-a6d9-038582089dda None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:01.072401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 611/1219] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:49:00 2022] GET /identity/v3/role_assignments?user.id=92937af60f80421094f4bdf6c7620283&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:03.238628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979bcb-8b59-443f-a4c0-c0d022277c05 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:03.239546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979bcb-8b59-443f-a4c0-c0d022277c05 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:03.239546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979bcb-8b59-443f-a4c0-c0d022277c05 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:03.241527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 609/1220] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:03.246822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:03.247526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:03.247526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:03.268574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:03.269801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:03.355468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 612/1221] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:03.634970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78979bcb-8b59-443f-a4c0-c0d022277c05 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:03.763731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78979bcb-8b59-443f-a4c0-c0d022277c05 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:03.803214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:03.803865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:03.804180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:03.804558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:03.816969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:49:03.822673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:03.823878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:03.825098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 610/1222] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:49:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:49:03.831200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:03.944307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efcbbe-7c53-4d32-935c-ea3ec1145cf5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:03.969383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:03.988061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:03.988670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:03.988950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:03.989222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:03.989883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] RBAC: Adding query filter params (name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:03.990187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:03.991571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:03.999639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 613/1223] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:49:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:04.005864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:04.052918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:04.112321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2532-afb1-46d3-b1f8-8391d2c61703 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:04.147628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:04.148260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:04.148551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:04.148813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:04.159653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:04.161089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:04.176751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:04.177941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:04.179074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:04.179074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:04.179074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:04.179074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 611/1224] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:04.183958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:04.297242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d7b33-d85d-43b1-88f0-8a8ce3ee837f None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:04.335407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:04.336154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:04.336455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:04.336731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:04.337387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:04.337717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:04.339196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:04.353126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 614/1225] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:04.359074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:04.468481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:04.476376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b535a-3375-4e49-8dc3-38f7f7b74f6e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:04.512175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:04.512807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:04.513089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:04.513361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:04.522070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:04.523550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:49:04.536175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:04.537360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:04.538617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 612/1226] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:04.543281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:04.598819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:04.661809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43a889-295b-4b26-8bbb-aa30fd2339a9 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:04.696523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:04.697218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:04.697574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:04.697867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:04.698553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:04.698874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:04.700295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:04.721692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 615/1227] 10.222.0.72 () {56 vars in 1143 bytes} [Mon Sep 26 17:49:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:04.727704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:04.841325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c32ecb-f67e-48eb-bf68-49f4b7817a19 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:04.879955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:04.880570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:04.880869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:04.881200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:04.893273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:04.894926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:04.910600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:04.911837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:04.913190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:04.913190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:04.913190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:04.913190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 613/1228] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:04.918322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:05.040087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01813d61-a72a-4ab9-a73d-f4e9a15bd615 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:05.042445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:05.077699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:05.078307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:05.078592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:05.078891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:05.079530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:05.079839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:05.081832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:05.093962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 616/1229] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:05.101304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:05.112436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:05.215901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea07031-37ab-4fca-98e8-a7f11514c51f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:05.253168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:05.254254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:05.254759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:05.255300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:05.265550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:05.267205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:49:05.281813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:05.283080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:05.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 614/1230] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:49:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:05.289841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:05.419348 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392bada-12ac-40a1-b8d7-af90d20601ee None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:05.458732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:05.459370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:05.459651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:05.459928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:05.460609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:05.460917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:05.462348 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b358a69-3ee6-4816-aece-639cd893c71f None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:05.475810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 617/1231] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:49:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:05.482649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:05.560219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:05.613229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f4b5-aba7-429b-afd6-787096351a6f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:05.654333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb306-f952-4096-9734-45affda71102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:05.655041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb306-f952-4096-9734-45affda71102 None admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:05.655328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb306-f952-4096-9734-45affda71102 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:05.655602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb306-f952-4096-9734-45affda71102 None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/92937af60f80421094f4bdf6c7620283/roles/5862dafdaa6d4353aefbaaa266832a76` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:05.672445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb306-f952-4096-9734-45affda71102 None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=92937af60f80421094f4bdf6c7620283, role_id=5862dafdaa6d4353aefbaaa266832a76)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:05.674868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb306-f952-4096-9734-45affda71102 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:05.690691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 615/1232] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:49:05 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/92937af60f80421094f4bdf6c7620283/roles/5862dafdaa6d4353aefbaaa266832a76 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:49:08.053891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498334a6-37e6-44dc-b07d-8f855e2a2efa None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:08.054458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498334a6-37e6-44dc-b07d-8f855e2a2efa None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:08.054602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498334a6-37e6-44dc-b07d-8f855e2a2efa None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:08.056346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 618/1233] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:08.061694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:08.062156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:08.062263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:08.082129 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:08.083372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:08.157245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 616/1234] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:49:08.456004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498334a6-37e6-44dc-b07d-8f855e2a2efa None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:08.483965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:08.572815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498334a6-37e6-44dc-b07d-8f855e2a2efa None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:08.617609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:08.618383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:08.618697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:08.619063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:08.631633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:49:08.637371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:08.645808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:08.646776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 619/1235] 10.222.0.72 () {56 vars in 1102 bytes} [Mon Sep 26 17:49:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:08.649006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:08.700357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:08.758066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4cf4f-a532-4b47-9a48-08330701e754 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:08.795415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:08.796061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:08.796368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:08.796688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:08.797356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:08.797663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:08.799281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:08.807827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 617/1236] 10.222.0.72 () {56 vars in 1103 bytes} [Mon Sep 26 17:49:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:08.813614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:08.901404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:08.922573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7caa42b-88f8-45eb-9a80-31b838374638 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:08.961943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:08.962572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:08.962861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:08.963146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:08.974656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:08.976142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:08.991904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:08.997732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:08.998901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:08.998901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:08.998901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:08.998901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 620/1237] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:08.999204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:09.112558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d7333-beaa-44c8-9374-7c3051a204ea None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:09.123015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:09.151183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:09.151800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:09.152083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:09.152402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:09.153140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:09.153502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:09.155051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:09.168138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 618/1238] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:09.174480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:09.284764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c62a7-f52f-48ff-8aaa-728d57c52a1e None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:09.322106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:09.322730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:09.323032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:09.323303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:09.326227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:09.332055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:09.333487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:49:09.348498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:09.350055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:09.351365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 621/1239] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:09.355816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:09.466933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893aac82-a152-404f-a34f-152cd424e37e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:09.503696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:09.504318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:09.504598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:09.504936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:09.506011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:09.506465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:09.507943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:09.526047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 619/1240] 10.222.0.72 () {56 vars in 1143 bytes} [Mon Sep 26 17:49:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:09.531776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:09.646043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72826e6-264f-4e8c-a149-1eded8fd1c91 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:09.686572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:09.687214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:09.687494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:09.687775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:09.770769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:09.778185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:09.779705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:09.881804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 622/1241] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:49:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:49:09.887687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:09.920257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:10.006088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1471b8-179b-4656-a2c6-2cb1ba27471c None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:10.044982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:10.045533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:10.045816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:10.046091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:10.057476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:10.058944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:10.074747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:10.076587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:10.077983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:10.077983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:10.077983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:10.077983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 620/1242] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:10.082913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:10.184974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:10.191766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e4a4-f026-4c0f-b3a9-6b40e5aca8cd None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:10.234951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:10.235629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:10.235990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:10.236327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:10.237218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:10.237594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:10.239267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:10.252264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 623/1243] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:10.258233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:10.347647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:10.372848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05ed6-5f41-45ed-ba53-daf93ab29539 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:10.412484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:10.413232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:10.413437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:10.413718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:10.424551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:10.427151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:49:10.447544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:10.450675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:10.451809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 621/1244] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:10.456342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:10.564973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d030cee-1daf-45e4-8b9b-de9104f87db3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:10.610262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:10.610919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:10.611211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:10.611855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:10.612145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:10.612456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:10.613916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f7a6d-82fd-4884-aa4c-1788111e8475 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:10.628055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 624/1245] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:10.634550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:10.756658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fc766-5b16-4285-b725-39f4ee7a7da0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:10.805563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:10.807587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:10.807587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:10.807587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:10.807587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] RBAC: Adding query filter params (role.id=5862dafdaa6d4353aefbaaa266832a76, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=92937af60f80421094f4bdf6c7620283) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:10.807959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:10.809109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9a058-f520-4df1-bd69-2ccf9c58f3ca None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:10.816799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 622/1246] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:49:10 2022] GET /identity/v3/role_assignments?user.id=92937af60f80421094f4bdf6c7620283&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=5862dafdaa6d4353aefbaaa266832a76 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:13.067932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939eede-bd25-40b1-8a1d-301975690973 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:13.068458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939eede-bd25-40b1-8a1d-301975690973 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:13.068458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939eede-bd25-40b1-8a1d-301975690973 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:13.070092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 625/1247] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:13.075428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:13.075861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:13.076125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:13.083792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:13.098551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:13.099941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:13.180667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 623/1248] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:13.459737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939eede-bd25-40b1-8a1d-301975690973 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:13.479930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:13.573463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939eede-bd25-40b1-8a1d-301975690973 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:13.618205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:13.619244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:13.619548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:13.619948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:13.635266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:49:13.641459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:13.643166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:13.644076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 626/1249] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:13.649893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:13.761820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8111cbb-c195-4cb9-8d83-8ed9bb927cf9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:13.800099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:13.807314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:13.807925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:13.808207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:13.808477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:13.809136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:13.809439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:13.810799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:13.819545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 624/1250] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:13.825647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:13.911530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:13.937048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e758827-f6bb-4783-9fc5-f61d584432a3 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:13.977315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:13.977960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:13.978245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:13.978525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:13.990530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:13.992349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:14.008967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:14.010377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:14.011612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:14.011612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:14.011612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:14.011612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 627/1251] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:14.016119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:14.129633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a99aef-f8cb-4cb8-be58-c016bba66c9e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:14.167161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:14.167787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:14.168064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:14.168335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:14.168989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:14.169292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:14.170687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:14.183575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 625/1252] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:14.189843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:14.299414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172559-989b-422a-89a1-aa3f1065b0ea None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:14.328305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:14.337704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:14.338344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:14.338624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:14.338930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:14.347335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:14.348806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:49:14.360985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:14.362212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:14.363436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:14.367215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 628/1253] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:14.368308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:14.428942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:14.476645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcbee4-6951-4037-937f-4a6ba7b46b6e None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:14.511380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:14.511994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:14.512271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:14.512542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:14.513267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:14.513577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:14.515030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:14.533227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 626/1254] 10.222.0.72 () {56 vars in 1143 bytes} [Mon Sep 26 17:49:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:14.539327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:14.656015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958fbfdc-0a77-413a-b474-74b18047e592 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:14.691694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:14.692304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:14.692585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:14.692854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:14.748329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:14.760990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:14.762427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:14.858276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 629/1255] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:49:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:49:14.864609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:14.960866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888de5e-d62a-40f6-ae7d-230445208999 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:14.974289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:14.996115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:14.996743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:14.997024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:14.997298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:15.007362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:15.008792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:15.023211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:15.024354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:15.025451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:15.025451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:15.025451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:15.025451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 627/1256] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:15.030108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:15.116847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:15.128861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcbf61-fb8d-4693-9212-f3ab0140c9d2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:15.162368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:15.163020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:15.163305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:15.163576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:15.164209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:15.164513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:15.166611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:15.179161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 630/1257] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:15.184784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:15.281179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0407a7-5f5a-4b53-a696-3536c03cc2e8 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:15.313845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:15.314487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:15.314764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:15.315054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:15.321690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:15.323145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:49:15.334519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:15.335613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:15.336725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 628/1258] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:15.341540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:15.442206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d331a4-09c5-4952-ab1d-6621cc7693be None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:15.474708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:15.475342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:15.475622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:15.475896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:15.476572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:15.476875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:15.478285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdb624-f9af-4ebf-9509-16bf0aa63129 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:15.490354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 631/1259] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:15.496398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:15.522542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:15.597568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bad54-b94a-4408-ab82-2d69f83fff52 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:15.637067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:15.637677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:15.638082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:15.638219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:15.638984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=92937af60f80421094f4bdf6c7620283) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:15.639273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:15.640696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68393f7-6be5-4647-9dc6-6d1a70534209 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:15.646769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 629/1260] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:49:15 2022] GET /identity/v3/role_assignments?user.id=92937af60f80421094f4bdf6c7620283&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:17.777766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c38c0-d7c7-4e84-ae90-a6e98c4b5311 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:17.778297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c38c0-d7c7-4e84-ae90-a6e98c4b5311 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:17.778297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c38c0-d7c7-4e84-ae90-a6e98c4b5311 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:17.779709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 632/1261] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:17.785507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:17.785936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:17.786057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:17.804011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:17.805194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:17.874948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 630/1262] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:18.143836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c38c0-d7c7-4e84-ae90-a6e98c4b5311 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:18.154569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:18.245288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c38c0-d7c7-4e84-ae90-a6e98c4b5311 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:18.279072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:18.279684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:18.279963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:18.280233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:18.291219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:49:18.297834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:18.299553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:18.300486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 633/1263] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:49:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:18.306278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:18.325547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:18.404165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de52ff-05eb-4ecf-b7cc-1d160b2706a2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:18.437799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:18.438399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:18.438690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:18.438996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:18.440502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:18.440802 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:18.442152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:18.450426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 631/1264] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:49:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:18.456177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:18.522634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:18.555977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d80ce5-3bfc-458f-b38f-dd043afa42e5 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:18.594766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:18.596346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:18.596773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:18.597058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:18.612567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:18.614291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:18.632022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:18.633040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:18.633963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:18.633963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:18.633963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:18.633963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 634/1265] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:18.639434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:18.723982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:18.742687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3b8e9-266f-4381-9e82-d4da67502b22 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:18.775801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:18.776405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:18.776685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:18.776957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:18.777591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:18.777894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:18.779280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:18.790830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 632/1266] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:18.796708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:18.895571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771583e4-3c8d-4a7f-b578-b0b3de48ba10 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:18.914439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:18.928620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:18.929234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:18.929514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:18.929785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:18.936981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:18.938412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:49:18.950134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:18.951328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:18.952630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 635/1267] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:18.957848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:19.057108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45bba-9661-4ff1-9507-aa0ed14e8e22 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:19.090707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:19.091334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:19.091614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:19.091886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:19.092606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:19.092914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:19.094323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:19.103422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:19.111895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 633/1268] 10.222.0.72 () {56 vars in 1143 bytes} [Mon Sep 26 17:49:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:19.117795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:19.216235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ace024-f05a-43f5-aff7-bac486073ec7 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:19.248842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:19.249380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:19.249661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:19.249932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:19.260853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:19.262298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:19.276715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:19.277839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:19.278784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:19.278784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:19.278784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:19.278784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 636/1269] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:19.284136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:19.380580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45145b11-2314-4cee-8e04-191ad903f1b1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:19.414663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:19.415293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:19.415575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:19.415877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:19.416511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:19.416812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:19.418190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:19.429160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 634/1270] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:19.434651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:19.466662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:19.533908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356753b-e9a8-4f7b-a679-ff3d03a43d87 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:19.566217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:19.566824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:19.567132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:19.567404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:19.574048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:19.575518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:49:19.587577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:19.588832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:19.590659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 637/1271] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:19.595829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:19.657909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:19.702435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db5067-e5e8-40fc-9c21-9c27c8d54ec0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:19.735941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:19.736668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:19.736944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:19.737224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:19.737907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:19.738213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:19.739651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccade0-99d7-4be0-a129-f4dc444d1593 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:19.751487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 635/1272] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:49:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:19.758013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:19.847921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:19.857342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef35a9-20d5-412a-ac80-f53c8c8f99e1 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:19.890828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905d0dd-8ca9-4ef5-a4e4-07b94391645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:19.891496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905d0dd-8ca9-4ef5-a4e4-07b94391645b None admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:19.891774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905d0dd-8ca9-4ef5-a4e4-07b94391645b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:19.892047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905d0dd-8ca9-4ef5-a4e4-07b94391645b None admin] PATH_INFO: `/v3/projects/54e4118bb98b467c9b698a23bba22007/users/92937af60f80421094f4bdf6c7620283/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:19.894034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905d0dd-8ca9-4ef5-a4e4-07b94391645b None admin] RBAC: Authorizing `identity:create_grant(project_id=54e4118bb98b467c9b698a23bba22007, user_id=92937af60f80421094f4bdf6c7620283, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:19.895480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905d0dd-8ca9-4ef5-a4e4-07b94391645b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:19.907625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 638/1273] 10.222.0.72 () {58 vars in 1439 bytes} [Mon Sep 26 17:49:19 2022] PUT /identity/v3/projects/54e4118bb98b467c9b698a23bba22007/users/92937af60f80421094f4bdf6c7620283/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:49:22.056821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dcd25-da5d-48ff-879f-bca906cba359 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:22.057310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dcd25-da5d-48ff-879f-bca906cba359 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:22.057400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dcd25-da5d-48ff-879f-bca906cba359 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:22.058699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 636/1274] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:22.064028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:22.064500 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:22.064613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:22.083443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:22.084625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:22.149600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 639/1275] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:22.421864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dcd25-da5d-48ff-879f-bca906cba359 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:22.521302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dcd25-da5d-48ff-879f-bca906cba359 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:22.537770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:22.553934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:22.554547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:22.554824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:22.555131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:22.565442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:49:22.570448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:22.571595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:22.572855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 637/1276] 10.222.0.72 () {56 vars in 1096 bytes} [Mon Sep 26 17:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:22.577534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:22.673850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:22.694658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ef5c9-2c9c-4d6f-a808-91a6f54c7065 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:22.728715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:22.729313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:22.729592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:22.729863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:22.730510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:22.730809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:22.732193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:22.740258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 640/1277] 10.222.0.72 () {56 vars in 1099 bytes} [Mon Sep 26 17:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:22.746125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:22.843957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4ce1a-3bdf-4d5c-be1a-2ba4c0d8d49f None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:22.876713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:22.877897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:22.878175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:22.878448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:22.888248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:22.889665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:22.903921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:22.905057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:22.906353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:22.906353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:22.906353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:22.906353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 638/1278] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:22.910820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:23.016607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88afdae-ca57-4eed-bd0a-5651ac2aa146 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:23.051665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:23.052263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:23.052543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:23.052816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:23.053446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:23.053747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:23.055137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:23.067342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 641/1279] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:23.073306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:23.105634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:23.172737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9c8ba-1ae8-4e97-9d46-248cc71c4a54 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:23.205840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:23.206435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:23.206716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:23.207015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] PATH_INFO: `/v3/users/placement` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:23.214033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:23.215475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:49:23.226186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:23.227383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:23.228565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 639/1280] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:23.234746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:23.238510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:23.334771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff53bf-14f9-4148-9d87-dfcdf9067ac2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:23.374180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:23.374790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:23.375100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:23.375373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:23.376094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:23.376401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:23.378469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:23.395880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 642/1281] 10.222.0.72 () {56 vars in 1143 bytes} [Mon Sep 26 17:49:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:23.401827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:23.492057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:23.502310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7cb27-65dc-43d3-8863-a202123a3fab None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:23.535190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:23.535811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:23.536093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:23.536368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:23.606009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:23.607434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:23.695100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:23.705670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 640/1282] 10.222.0.72 () {58 vars in 1282 bytes} [Mon Sep 26 17:49:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:49:23.711503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:23.774824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:23.813501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf670-3b4e-4186-abf0-8b17c1e27656 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:23.846967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:23.847587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:23.847871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:23.848147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:23.857842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:23.859308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:49:23.874062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:23.875226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:49:23.876186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:49:23.876186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:49:23.876186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:23.876186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 643/1283] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:23.882191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:23.980039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3ac3c-5a25-41c8-901d-c8a7f373ea20 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:24.014027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:24.014632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:24.014945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:24.015221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:24.015862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:24.016167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:24.017557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:24.028555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 641/1284] 10.222.0.72 () {56 vars in 1109 bytes} [Mon Sep 26 17:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:24.034719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:24.132611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51be8aa-cd53-41c2-b34c-3dfc927be548 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:24.145739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:24.166391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:24.167037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:24.167321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:24.167596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:24.174778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:24.176239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:49:24.187367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:24.188408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: ERROR keystone.server.flask.application Sep 26 17:49:24.189666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 644/1285] 10.222.0.72 () {56 vars in 1111 bytes} [Mon Sep 26 17:49:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:24.194601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:24.244232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:24.293038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3fbb-e3d9-4181-9a94-95dc28264bb1 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:24.325998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:24.326593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:24.326902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:24.327179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:24.327860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:24.328167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:24.329578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f11b03-674b-462d-b63a-db3860f785fe None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:24.341782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 642/1286] 10.222.0.72 () {56 vars in 1148 bytes} [Mon Sep 26 17:49:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:24.348399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:24.445424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca289678-1bc5-4a8e-8610-5b01d42163ea None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:24.478025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:24.478606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:24.478904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:24.479175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:24.479881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] RBAC: Adding query filter params (role.id=3c63afd56d04499eb8c3c56d8aa62dba, scope.project.id=54e4118bb98b467c9b698a23bba22007, user.id=92937af60f80421094f4bdf6c7620283) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:24.480184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:24.481596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8835-4664-4091-866c-74a1e8a5bb54 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:24.487231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 645/1287] 10.222.0.72 () {56 vars in 1374 bytes} [Mon Sep 26 17:49:24 2022] GET /identity/v3/role_assignments?user.id=92937af60f80421094f4bdf6c7620283&scope.project.id=54e4118bb98b467c9b698a23bba22007&role.id=3c63afd56d04499eb8c3c56d8aa62dba => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:26.724883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf87841-88c5-429e-a2b8-37a672a7c5bf None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:26.725442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf87841-88c5-429e-a2b8-37a672a7c5bf None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:26.725442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf87841-88c5-429e-a2b8-37a672a7c5bf None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:26.726771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 643/1288] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:26.731804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:26.732227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:26.732358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:26.750225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:26.751443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:26.780416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:26.817848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 646/1289] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:27.083600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf87841-88c5-429e-a2b8-37a672a7c5bf None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:27.182055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf87841-88c5-429e-a2b8-37a672a7c5bf None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:27.192553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:27.216186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:27.216799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:27.217074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:27.217341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:27.217915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:27.219293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:27.228635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:27.229870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:27.230772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 644/1290] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:49:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:27.235835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:27.332977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48579228-6cd4-414b-b58f-8c735df20a15 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:27.365172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:27.365765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:27.366041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:27.366308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:27.366949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:27.367251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:27.368610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69284166-6239-46c2-b676-7c295c9d4888 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:27.373190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 647/1291] 10.222.0.72 () {56 vars in 1116 bytes} [Mon Sep 26 17:49:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:27.379797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:27.476612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66136db-4416-4055-9943-9dc727fa9719 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:27.509378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:27.509887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:27.510164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:27.510431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] PATH_INFO: `/v3/services` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:27.511074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:27.511375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:27.512713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c23294-deee-4101-9153-67e0379f1657 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:27.516932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 645/1292] 10.222.0.72 () {56 vars in 1116 bytes} [Mon Sep 26 17:49:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:49:29.660738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba829c5b-6d5f-469d-b7be-91f077f60fb5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:29.661312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba829c5b-6d5f-469d-b7be-91f077f60fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:29.661312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba829c5b-6d5f-469d-b7be-91f077f60fb5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:29.663084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 648/1293] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:49:29.668269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99c97-4ea8-49e4-bcb6-766025719350 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:29.668665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99c97-4ea8-49e4-bcb6-766025719350 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:29.668821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99c97-4ea8-49e4-bcb6-766025719350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:29.687164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8da99c97-4ea8-49e4-bcb6-766025719350 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:29.688346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8da99c97-4ea8-49e4-bcb6-766025719350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:29.717523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:29.754755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 646/1294] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:30.022990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba829c5b-6d5f-469d-b7be-91f077f60fb5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:30.054298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:30.124291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba829c5b-6d5f-469d-b7be-91f077f60fb5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:30.158110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a51c20-0d12-4bbd-ba1e-56a720fed7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:30.158644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a51c20-0d12-4bbd-ba1e-56a720fed7c8 None admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:30.158956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a51c20-0d12-4bbd-ba1e-56a720fed7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:30.159232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a51c20-0d12-4bbd-ba1e-56a720fed7c8 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:30.159886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a51c20-0d12-4bbd-ba1e-56a720fed7c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:30.161257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a51c20-0d12-4bbd-ba1e-56a720fed7c8 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:30.176555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 649/1295] 10.222.0.72 () {60 vars in 1141 bytes} [Mon Sep 26 17:49:30 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:49:32.323051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4100-faa4-4375-bb7c-f97af36c04a6 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:32.323616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4100-faa4-4375-bb7c-f97af36c04a6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:32.323616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4100-faa4-4375-bb7c-f97af36c04a6 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:32.324866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 647/1296] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:32.330771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:32.331221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:32.331349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:32.349289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:32.350477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:32.422121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 650/1297] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:32.725107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4100-faa4-4375-bb7c-f97af36c04a6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:32.824271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4100-faa4-4375-bb7c-f97af36c04a6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:32.856928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:32.857980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:32.858461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:32.858963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:32.859983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:32.863121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:32.873870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:32.875156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:32.876334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 648/1298] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:49:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:32.881884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:32.899585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:32.983501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e61bff-9d79-4c7a-a38a-1f52474ddbe9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:33.020658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:33.021285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:33.021566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:33.021878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:33.022536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:33.022870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:33.024254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:33.029682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 651/1299] 10.222.0.72 () {56 vars in 1116 bytes} [Mon Sep 26 17:49:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:33.036513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:33.067972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:33.143334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e66fb7-a1cc-47ff-9f5e-0fea4952d4c4 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:33.190958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:33.191571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:33.191854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:33.192130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:33.192831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] RBAC: Adding query filter params (interface=public, service_id=dae9fc3afa7f438286876597e889e7b5, region_id=RegionOne) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:33.193147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:33.194514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386dcb-277a-4d84-a344-84040fd8faf7 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:33.199864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 649/1300] 10.222.0.72 () {56 vars in 1251 bytes} [Mon Sep 26 17:49:33 2022] GET /identity/v3/endpoints?service_id=dae9fc3afa7f438286876597e889e7b5&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:33.207599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:33.303319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:33.323780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be334304-c4a1-4d68-b2b1-f464fec24da2 None admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:33.363454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b3643-575e-4cbd-bff1-d7fa00391463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:33.364135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b3643-575e-4cbd-bff1-d7fa00391463 None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:33.364327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b3643-575e-4cbd-bff1-d7fa00391463 None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:33.364603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b3643-575e-4cbd-bff1-d7fa00391463 None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:33.365211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b3643-575e-4cbd-bff1-d7fa00391463 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:33.367074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b3643-575e-4cbd-bff1-d7fa00391463 None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:33.375101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 652/1301] 10.222.0.72 () {56 vars in 1087 bytes} [Mon Sep 26 17:49:33 2022] GET /identity/v3/services => generated 2191 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:49:35.592516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377342d1-0973-40c4-9a2d-3e0879e2a944 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:35.593082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377342d1-0973-40c4-9a2d-3e0879e2a944 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:35.593082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377342d1-0973-40c4-9a2d-3e0879e2a944 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:35.594415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 650/1302] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:49:35.599569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:35.599986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:35.600112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:35.618321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:49:35.619526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:49:35.684076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 653/1303] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:49:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:49:35.954044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377342d1-0973-40c4-9a2d-3e0879e2a944 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:36.049823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:36.056813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377342d1-0973-40c4-9a2d-3e0879e2a944 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:36.089199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:36.089847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:36.090128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:36.090398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:36.091003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:36.092355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:49:36.101406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:49:36.102383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application with Lock( Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: ERROR keystone.server.flask.application Sep 26 17:49:36.103250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 651/1304] 10.222.0.72 () {56 vars in 1117 bytes} [Mon Sep 26 17:49:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:49:36.108503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:36.183586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:36.206699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7ae37-07e4-4ee4-b794-553670c662c8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:36.240286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:36.240880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:36.241161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:36.241433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] PATH_INFO: `/v3/services` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:36.242057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:49:36.242644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:36.244039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec94f6d-08f9-4ced-9943-51b82efa225b None admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:36.249094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 654/1305] 10.222.0.72 () {56 vars in 1116 bytes} [Mon Sep 26 17:49:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:49:36.255627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:49:36.353275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da533fe-76c2-430b-8f9b-1d36302123b0 None admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:49:36.386957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a40e43-7cd9-46ed-b818-c5f6be5f451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:49:36.387552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40e43-7cd9-46ed-b818-c5f6be5f451e None admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:49:36.387830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40e43-7cd9-46ed-b818-c5f6be5f451e None admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:49:36.388100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a40e43-7cd9-46ed-b818-c5f6be5f451e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:49:36.388767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a40e43-7cd9-46ed-b818-c5f6be5f451e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:49:36.390124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a40e43-7cd9-46ed-b818-c5f6be5f451e None admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:49:36.408948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 652/1306] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:49:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:49:46.246324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=30 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:49:46.404718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:41.874769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3326-b2f2-4d97-8fa4-f8bd3217df38 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:41.875206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3326-b2f2-4d97-8fa4-f8bd3217df38 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:41.875294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c3326-b2f2-4d97-8fa4-f8bd3217df38 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:41.876897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 655/1307] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:41.882863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e99278-f59e-4471-911e-eadf3a49706e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:41.883330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e99278-f59e-4471-911e-eadf3a49706e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:41.883517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e99278-f59e-4471-911e-eadf3a49706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:42.061879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-56e99278-f59e-4471-911e-eadf3a49706e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:42.063310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-56e99278-f59e-4471-911e-eadf3a49706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:42.190894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 653/1308] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:50:42.485455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84ba87-3e51-4a79-be84-831cef6ea7fd None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:42.485740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84ba87-3e51-4a79-be84-831cef6ea7fd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:42.486287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84ba87-3e51-4a79-be84-831cef6ea7fd None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:42.488650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 656/1309] 10.222.0.72 () {54 vars in 964 bytes} [Mon Sep 26 17:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:42.494706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10028c-5fa5-4ebb-a8b9-aa5d07e2f323 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:42.495106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10028c-5fa5-4ebb-a8b9-aa5d07e2f323 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:42.495482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10028c-5fa5-4ebb-a8b9-aa5d07e2f323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:42.584269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2a10028c-5fa5-4ebb-a8b9-aa5d07e2f323 None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:42.585373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2a10028c-5fa5-4ebb-a8b9-aa5d07e2f323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:42.608480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:42.671379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 654/1310] 10.222.0.72 () {58 vars in 1061 bytes} [Mon Sep 26 17:50:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:50:42.680593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84ba87-3e51-4a79-be84-831cef6ea7fd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:50:42.682332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2f84ba87-3e51-4a79-be84-831cef6ea7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:42.797342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84ba87-3e51-4a79-be84-831cef6ea7fd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:50:42.875975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:50:42.876600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:42.876866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:42.877034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:42.879522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:42.939191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:43.017582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:50:43.019724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6b303-f2ee-422d-a4ca-f38063cfc131 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:50:43.041154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 657/1311] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:50:45.544711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a44a3e-3026-4efb-8338-06ed8c5f9e06 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:45.545266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a44a3e-3026-4efb-8338-06ed8c5f9e06 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:45.545266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a44a3e-3026-4efb-8338-06ed8c5f9e06 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:45.546629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 655/1312] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:45.552080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cdbeb-1391-450b-b2e3-b8f446eb8e03 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:45.552512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cdbeb-1391-450b-b2e3-b8f446eb8e03 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:45.552626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cdbeb-1391-450b-b2e3-b8f446eb8e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:45.623611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5c6cdbeb-1391-450b-b2e3-b8f446eb8e03 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:45.625340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c6cdbeb-1391-450b-b2e3-b8f446eb8e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:45.727244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 658/1313] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:50:46.014559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a44a3e-3026-4efb-8338-06ed8c5f9e06 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:50:46.016119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-40a44a3e-3026-4efb-8338-06ed8c5f9e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:46.124610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a44a3e-3026-4efb-8338-06ed8c5f9e06 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:50:46.197073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:46.199250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:50:46.199931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:46.200446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:46.200937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:46.202786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:46.312392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:50:46.313752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c67d37-35c6-4b19-bb32-8b2e2374bb85 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:50:46.332698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 656/1314] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:50:48.853552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340b09-88b6-4a31-8c07-ec1c6f7778f1 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:48.854092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340b09-88b6-4a31-8c07-ec1c6f7778f1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:48.854092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340b09-88b6-4a31-8c07-ec1c6f7778f1 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:48.855500 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 659/1315] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:48.860628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a60d6f-749e-440d-8d16-e8f391d76664 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:48.861057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a60d6f-749e-440d-8d16-e8f391d76664 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:48.861169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a60d6f-749e-440d-8d16-e8f391d76664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:48.927385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a5a60d6f-749e-440d-8d16-e8f391d76664 None None] MFA Rules not processed for user `0b672c7685154ab1a019a0a63150dcf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:48.928599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a5a60d6f-749e-440d-8d16-e8f391d76664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:48.982193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:49.028332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 657/1316] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:50:49.332803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c282-b4b7-4fe7-b7b6-f338c70e1c03 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:49.333220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c282-b4b7-4fe7-b7b6-f338c70e1c03 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:49.333510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c282-b4b7-4fe7-b7b6-f338c70e1c03 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:49.334887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 660/1317] 10.222.0.72 () {54 vars in 964 bytes} [Mon Sep 26 17:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:49.340809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73373-d275-42bd-8b76-000c8ba03e9d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:49.341095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73373-d275-42bd-8b76-000c8ba03e9d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:49.341373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a73373-d275-42bd-8b76-000c8ba03e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:49.418567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-56a73373-d275-42bd-8b76-000c8ba03e9d None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:49.419828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-56a73373-d275-42bd-8b76-000c8ba03e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:49.498505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 658/1318] 10.222.0.72 () {58 vars in 1061 bytes} [Mon Sep 26 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:50:49.507914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c282-b4b7-4fe7-b7b6-f338c70e1c03 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:50:49.509426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-34a8c282-b4b7-4fe7-b7b6-f338c70e1c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:49.519047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:49.613669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c282-b4b7-4fe7-b7b6-f338c70e1c03 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:50:49.686751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:50:49.687377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:49.687654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:49.687923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:49.689509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:49.753458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:49.800319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:50:49.801783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51504d34-9efb-44e8-b1d1-a29a3699c4a9 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:50:49.820112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 661/1319] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:50:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:50:52.930652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5010-40e5-451b-8f06-1a77c3a05ca6 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:52.931227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5010-40e5-451b-8f06-1a77c3a05ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:52.931227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5010-40e5-451b-8f06-1a77c3a05ca6 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:52.932595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 659/1320] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:52.938043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f271641-3170-49c3-bfca-29c0299d0041 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:52.938419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f271641-3170-49c3-bfca-29c0299d0041 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:52.938602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f271641-3170-49c3-bfca-29c0299d0041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:53.006950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5f271641-3170-49c3-bfca-29c0299d0041 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:53.008164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5f271641-3170-49c3-bfca-29c0299d0041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:53.103124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 662/1321] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:50:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:50:53.389953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5010-40e5-451b-8f06-1a77c3a05ca6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:50:53.391566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-774c5010-40e5-451b-8f06-1a77c3a05ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:53.441706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:53.495494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5010-40e5-451b-8f06-1a77c3a05ca6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:50:53.567581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:50:53.568183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:53.568463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:53.568735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:53.570316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:53.668673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:53.681985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:50:53.683457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc0aa-0eed-49c2-8001-1e6e6919c16a service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:50:53.701362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 660/1322] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:50:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:50:56.602253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7656a-d587-4da1-8a80-33eb05b28bec None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:56.602813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7656a-d587-4da1-8a80-33eb05b28bec None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:56.602813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7656a-d587-4da1-8a80-33eb05b28bec None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:56.604205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 663/1323] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:56.609437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631a8dc-d6cf-40c7-b412-0c65d766b812 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:56.609831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631a8dc-d6cf-40c7-b412-0c65d766b812 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:56.610093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631a8dc-d6cf-40c7-b412-0c65d766b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:56.677817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8631a8dc-d6cf-40c7-b412-0c65d766b812 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:56.679077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8631a8dc-d6cf-40c7-b412-0c65d766b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:56.787487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 661/1324] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:50:57.071714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7656a-d587-4da1-8a80-33eb05b28bec None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:50:57.073233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-80e7656a-d587-4da1-8a80-33eb05b28bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:57.093342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:57.181205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7656a-d587-4da1-8a80-33eb05b28bec None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:50:57.275572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:50:57.276442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:57.276853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:57.277274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:57.279283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:57.354254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:50:57.392169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:50:57.393647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6474cf-75e8-47a2-9730-95d048c59413 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:50:57.411982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 664/1325] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:50:59.646362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccba32-9225-4f17-b7aa-f6cad22599a9 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:59.646945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccba32-9225-4f17-b7aa-f6cad22599a9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:59.646945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccba32-9225-4f17-b7aa-f6cad22599a9 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:59.652308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 662/1326] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:50:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:50:59.653857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48109157-bb2c-4413-b83f-2b9d4aad3019 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:50:59.654289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48109157-bb2c-4413-b83f-2b9d4aad3019 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:50:59.654440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48109157-bb2c-4413-b83f-2b9d4aad3019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:50:59.730362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-48109157-bb2c-4413-b83f-2b9d4aad3019 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:50:59.731780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48109157-bb2c-4413-b83f-2b9d4aad3019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:50:59.848007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 665/1327] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:00.144487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccba32-9225-4f17-b7aa-f6cad22599a9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:00.146108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-60ccba32-9225-4f17-b7aa-f6cad22599a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:00.210722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:00.258201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccba32-9225-4f17-b7aa-f6cad22599a9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:00.345476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:00.346113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:00.346422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:00.346701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:00.348328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:00.456786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:00.459477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:00.461352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b672e74-19b2-469f-8d06-b87cee229775 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:00.480006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 663/1328] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:02.740904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68032e-6976-4a8b-bcb8-ad6cdcaf4938 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:02.741430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68032e-6976-4a8b-bcb8-ad6cdcaf4938 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:02.741430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68032e-6976-4a8b-bcb8-ad6cdcaf4938 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:02.742751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 666/1329] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:02.748153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf98179-1ebf-4df4-a9c2-f7f518fb6d51 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:02.748582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf98179-1ebf-4df4-a9c2-f7f518fb6d51 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:02.748694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf98179-1ebf-4df4-a9c2-f7f518fb6d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:02.816560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-acf98179-1ebf-4df4-a9c2-f7f518fb6d51 None None] MFA Rules not processed for user `0b672c7685154ab1a019a0a63150dcf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:02.817805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-acf98179-1ebf-4df4-a9c2-f7f518fb6d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:02.917365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 664/1330] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:03.215599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68032e-6976-4a8b-bcb8-ad6cdcaf4938 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:03.217143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9d68032e-6976-4a8b-bcb8-ad6cdcaf4938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:03.252231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:03.336958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68032e-6976-4a8b-bcb8-ad6cdcaf4938 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:03.420217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:03.420825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:03.421103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:03.421372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:03.422998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:03.505373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:03.539400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:03.540850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa94d11-864e-445f-970f-01940a026c0d service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:03.559920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 667/1331] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:05.973916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a259c7-f143-4f1f-86d6-0caebb0d5f72 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:05.974479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a259c7-f143-4f1f-86d6-0caebb0d5f72 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:05.974479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a259c7-f143-4f1f-86d6-0caebb0d5f72 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:05.975866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 665/1332] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:05.981287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec8d76-3346-44d9-8e72-a0c6aad170df None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:05.981728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec8d76-3346-44d9-8e72-a0c6aad170df None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:05.981837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec8d76-3346-44d9-8e72-a0c6aad170df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:06.047254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-98ec8d76-3346-44d9-8e72-a0c6aad170df None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:06.048466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-98ec8d76-3346-44d9-8e72-a0c6aad170df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:06.145012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 668/1333] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:06.436855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a259c7-f143-4f1f-86d6-0caebb0d5f72 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:06.438408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-51a259c7-f143-4f1f-86d6-0caebb0d5f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:06.507509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:06.540887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a259c7-f143-4f1f-86d6-0caebb0d5f72 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:06.613598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:06.614197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:06.614471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:06.614736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:06.616377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:06.722883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:06.724318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0be52-a26c-45d8-a1bc-c9901643b516 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:06.733976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:06.743379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 666/1334] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:08.954334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de27357-e98a-4384-9a81-b9db275fa04c None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:08.954946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de27357-e98a-4384-9a81-b9db275fa04c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:08.954946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de27357-e98a-4384-9a81-b9db275fa04c None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:08.957011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 669/1335] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:08.962198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac53dd-259d-4545-9a3a-2e98142a2b4d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:08.962610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac53dd-259d-4545-9a3a-2e98142a2b4d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:08.962755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac53dd-259d-4545-9a3a-2e98142a2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:09.028782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-81ac53dd-259d-4545-9a3a-2e98142a2b4d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:09.029998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-81ac53dd-259d-4545-9a3a-2e98142a2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:09.130375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 667/1336] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:09.420086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de27357-e98a-4384-9a81-b9db275fa04c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:09.421664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7de27357-e98a-4384-9a81-b9db275fa04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:09.462516 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:09.527391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de27357-e98a-4384-9a81-b9db275fa04c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:09.600880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:09.601484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:09.601764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:09.602036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:09.603668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:09.688821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:09.714995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:09.716434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f7590-cfe1-4014-b3ee-44a90c2aebfd service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:09.735349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 670/1337] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:12.348111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85885-b68d-4e02-90c1-842cf26b3a20 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:12.348638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85885-b68d-4e02-90c1-842cf26b3a20 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:12.348638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a85885-b68d-4e02-90c1-842cf26b3a20 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:12.349940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 668/1338] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:12.355315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ab12-49b7-421c-aaf4-94a58ff5897d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:12.355716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ab12-49b7-421c-aaf4-94a58ff5897d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:12.355858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ab12-49b7-421c-aaf4-94a58ff5897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:12.421422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5d27ab12-49b7-421c-aaf4-94a58ff5897d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:12.422636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5d27ab12-49b7-421c-aaf4-94a58ff5897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:12.518727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 671/1339] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:12.806231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a85885-b68d-4e02-90c1-842cf26b3a20 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:12.807834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-74a85885-b68d-4e02-90c1-842cf26b3a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:12.885976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:12.914100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a85885-b68d-4e02-90c1-842cf26b3a20 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:12.987580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:12.988134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:12.988408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:12.988676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:12.990251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:13.108797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:13.110145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07d8e1-a8f0-40dd-9b04-ee8d1251fff4 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:13.129152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 669/1340] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:17.404751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd800f8b-e445-43f7-b3dd-95996eca2c50 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:17.405302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd800f8b-e445-43f7-b3dd-95996eca2c50 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:17.405302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd800f8b-e445-43f7-b3dd-95996eca2c50 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:17.406712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 672/1341] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:17.412742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cd75c-0db7-4276-a7a1-911b9ae241e6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:17.413187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cd75c-0db7-4276-a7a1-911b9ae241e6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:17.413303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cd75c-0db7-4276-a7a1-911b9ae241e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:17.418711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:17.487037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4d3cd75c-0db7-4276-a7a1-911b9ae241e6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:17.488459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4d3cd75c-0db7-4276-a7a1-911b9ae241e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:17.591835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 670/1342] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:17.881234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd800f8b-e445-43f7-b3dd-95996eca2c50 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:17.882884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bd800f8b-e445-43f7-b3dd-95996eca2c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:17.934065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:17.991611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd800f8b-e445-43f7-b3dd-95996eca2c50 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:18.067831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:18.068380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:18.068679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:18.068949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:18.070530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:18.166905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:18.184082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:18.185515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ad471-2c6e-48f8-ad77-2c1907f32ed7 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:18.203733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 673/1343] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:20.999882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdd4e3-8e49-45be-8d65-da0d743c16a7 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:21.000378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdd4e3-8e49-45be-8d65-da0d743c16a7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:21.000462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdd4e3-8e49-45be-8d65-da0d743c16a7 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:21.001789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 671/1344] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:21.007603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4b48-811a-4e4a-a69c-e86dc4ff2d00 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:21.008023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4b48-811a-4e4a-a69c-e86dc4ff2d00 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:21.008160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4b48-811a-4e4a-a69c-e86dc4ff2d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:21.075461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f50e4b48-811a-4e4a-a69c-e86dc4ff2d00 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:21.076656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f50e4b48-811a-4e4a-a69c-e86dc4ff2d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:21.174231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 674/1345] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:21.459844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdd4e3-8e49-45be-8d65-da0d743c16a7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:21.461374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-13cdd4e3-8e49-45be-8d65-da0d743c16a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:21.554098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:21.571048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdd4e3-8e49-45be-8d65-da0d743c16a7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:21.659821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:21.660422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:21.660653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:21.660921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:21.662518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:21.773490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:21.774959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d305754-e2b8-4673-b36c-250826790cf2 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:21.792965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 672/1346] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:26.221937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650b866-c3fc-4475-abde-beb1954750fd None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:26.222414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650b866-c3fc-4475-abde-beb1954750fd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:26.222524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650b866-c3fc-4475-abde-beb1954750fd None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:26.223917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 675/1347] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:26.229464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96776336-1d42-4eff-b9d8-91e36f9a2ebd None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:26.229892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96776336-1d42-4eff-b9d8-91e36f9a2ebd None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:26.230161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96776336-1d42-4eff-b9d8-91e36f9a2ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:26.246438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:26.300698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-96776336-1d42-4eff-b9d8-91e36f9a2ebd None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:26.301988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-96776336-1d42-4eff-b9d8-91e36f9a2ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:26.397739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 673/1348] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:51:26.683691 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650b866-c3fc-4475-abde-beb1954750fd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:26.685243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b650b866-c3fc-4475-abde-beb1954750fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:26.746445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:26.792666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650b866-c3fc-4475-abde-beb1954750fd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:26.865756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:26.866356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:26.866637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:26.866939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:26.868540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:26.976414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:26.980742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:26.982497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c53b97-182a-4afc-8506-8886d12f99a1 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:27.003283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 676/1349] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:32.005245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6472b1-2dfe-4047-bc13-dcf4264da8c7 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:32.005711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6472b1-2dfe-4047-bc13-dcf4264da8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:32.005808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6472b1-2dfe-4047-bc13-dcf4264da8c7 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:32.007805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 674/1350] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 17:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:32.013182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03773dc-ecc0-4c2c-bda8-8ae488f83559 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:32.013547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03773dc-ecc0-4c2c-bda8-8ae488f83559 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:32.013644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03773dc-ecc0-4c2c-bda8-8ae488f83559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:32.149577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a03773dc-ecc0-4c2c-bda8-8ae488f83559 None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:32.150685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a03773dc-ecc0-4c2c-bda8-8ae488f83559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:32.268782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 677/1351] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 17:51:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:36.369087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc31cc-f3d9-41b3-ac29-5789770511f7 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:36.369691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc31cc-f3d9-41b3-ac29-5789770511f7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:36.369691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc31cc-f3d9-41b3-ac29-5789770511f7 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:36.371293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 675/1352] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 17:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:36.375874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a3ba1-dbd9-4103-9af9-15b7ade1512a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:36.376288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a3ba1-dbd9-4103-9af9-15b7ade1512a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:36.376494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a3ba1-dbd9-4103-9af9-15b7ade1512a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:36.399227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=59 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:36.434079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:36.458638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-621a3ba1-dbd9-4103-9af9-15b7ade1512a None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:36.459884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-621a3ba1-dbd9-4103-9af9-15b7ade1512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:36.575556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 678/1353] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 17:51:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:51:38.101530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37917926-461e-4bf4-b7bb-410fc55e6aa0 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:38.102539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37917926-461e-4bf4-b7bb-410fc55e6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:38.102539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37917926-461e-4bf4-b7bb-410fc55e6aa0 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:38.105438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 676/1354] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 17:51:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:38.110891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccf19f-8ffe-4e44-a417-fb7c887940a3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:38.114193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccf19f-8ffe-4e44-a417-fb7c887940a3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:38.114193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccf19f-8ffe-4e44-a417-fb7c887940a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:38.195964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7fccf19f-8ffe-4e44-a417-fb7c887940a3 None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:38.197721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7fccf19f-8ffe-4e44-a417-fb7c887940a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:38.240804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:38.306878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 679/1355] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 17:51:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:41.990390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0b687-6f34-4707-bd7a-673932238511 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:41.990898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0b687-6f34-4707-bd7a-673932238511 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:41.991134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0b687-6f34-4707-bd7a-673932238511 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:41.992567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 677/1356] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:41.997973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd4ddd-7d71-4d03-96ee-51779ce735f5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:41.998451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd4ddd-7d71-4d03-96ee-51779ce735f5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:41.998886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd4ddd-7d71-4d03-96ee-51779ce735f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:42.078907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e8fd4ddd-7d71-4d03-96ee-51779ce735f5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:42.080700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e8fd4ddd-7d71-4d03-96ee-51779ce735f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:42.200912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 680/1357] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:42.507253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c38d-6248-4838-a5db-0cdc048cc59d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:42.507760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c38d-6248-4838-a5db-0cdc048cc59d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:42.508200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c38d-6248-4838-a5db-0cdc048cc59d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:42.510227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 678/1358] 10.222.0.72 () {54 vars in 961 bytes} [Mon Sep 26 17:51:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:42.518165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d98e1-a401-4f6a-8f3f-01f44eb8226f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:42.519422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d98e1-a401-4f6a-8f3f-01f44eb8226f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:42.521382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d98e1-a401-4f6a-8f3f-01f44eb8226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:42.540270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:42.606306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1b1d98e1-a401-4f6a-8f3f-01f44eb8226f None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:42.607630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1b1d98e1-a401-4f6a-8f3f-01f44eb8226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:42.740151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 681/1359] 10.222.0.72 () {58 vars in 1058 bytes} [Mon Sep 26 17:51:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:51:42.747887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034c38d-6248-4838-a5db-0cdc048cc59d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:42.749424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4034c38d-6248-4838-a5db-0cdc048cc59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:42.907221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034c38d-6248-4838-a5db-0cdc048cc59d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:42.991238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:43.039714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:43.040426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:43.040835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:43.041124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:43.042818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:43.157070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:43.159552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595aabb-59cd-4f8d-8168-336bf2285939 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:43.179319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 679/1360] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:45.582953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75072-f5b8-4cc4-a093-7ed15b947772 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:45.583900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75072-f5b8-4cc4-a093-7ed15b947772 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:45.583900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75072-f5b8-4cc4-a093-7ed15b947772 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:45.592654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ac942-8c0a-4e4d-b2ad-10ba3baaf332 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:45.593264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ac942-8c0a-4e4d-b2ad-10ba3baaf332 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:45.593264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ac942-8c0a-4e4d-b2ad-10ba3baaf332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:45.599096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 682/1361] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:45 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:45.673355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-5e5ac942-8c0a-4e4d-b2ad-10ba3baaf332 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:45.674673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5e5ac942-8c0a-4e4d-b2ad-10ba3baaf332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:45.719317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:45.785926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 680/1362] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:46.089681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75072-f5b8-4cc4-a093-7ed15b947772 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:46.091563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5ab75072-f5b8-4cc4-a093-7ed15b947772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:46.167022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:46.283697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75072-f5b8-4cc4-a093-7ed15b947772 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:46.429380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:46.430051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:46.430361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:46.430635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:46.432514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:46.455894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:46.533162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:46.535692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee20cba-5409-4c63-92ea-c3e4a3055c15 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:46.555837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 683/1363] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:49.084155 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940270-953c-47d6-b5e0-bceab089c169 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:49.084697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940270-953c-47d6-b5e0-bceab089c169 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:49.084697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940270-953c-47d6-b5e0-bceab089c169 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:49.086067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 681/1364] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:49.091593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283fb19-06dd-4b98-a407-3bbbe84e9ec3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:49.092009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283fb19-06dd-4b98-a407-3bbbe84e9ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:49.092197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283fb19-06dd-4b98-a407-3bbbe84e9ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:49.163729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a283fb19-06dd-4b98-a407-3bbbe84e9ec3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:49.164956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a283fb19-06dd-4b98-a407-3bbbe84e9ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:49.266309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:49.273606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 684/1365] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:49.569488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088798cf-0d32-4670-8b37-0c69c0cfbfa2 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:49.569787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088798cf-0d32-4670-8b37-0c69c0cfbfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:49.570068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088798cf-0d32-4670-8b37-0c69c0cfbfa2 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:49.571839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 682/1366] 10.222.0.72 () {54 vars in 961 bytes} [Mon Sep 26 17:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:49.577496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806cf3f-0e26-4456-ade1-acd34eb1e1af None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:49.577956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806cf3f-0e26-4456-ade1-acd34eb1e1af None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:49.578347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806cf3f-0e26-4456-ade1-acd34eb1e1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:49.660923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6806cf3f-0e26-4456-ade1-acd34eb1e1af None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:49.662216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6806cf3f-0e26-4456-ade1-acd34eb1e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:49.788098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 685/1367] 10.222.0.72 () {58 vars in 1058 bytes} [Mon Sep 26 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:49.796902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088798cf-0d32-4670-8b37-0c69c0cfbfa2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:49.798407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-088798cf-0d32-4670-8b37-0c69c0cfbfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:49.958739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088798cf-0d32-4670-8b37-0c69c0cfbfa2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:49.986329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:50.090505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:50.091231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:50.091417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:50.091692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:50.093316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:50.198091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:50.200016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66828c1f-f3f5-43d0-8169-8a5bd5087df7 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:50.220438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 683/1368] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:51:50.290806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa3a2c-c044-4df6-8069-a30585ec5db8 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:50.291346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa3a2c-c044-4df6-8069-a30585ec5db8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:50.291428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa3a2c-c044-4df6-8069-a30585ec5db8 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:50.292910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 686/1369] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:50.298061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15162ce-3100-40db-a9ec-5b1ab5028726 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:50.298574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15162ce-3100-40db-a9ec-5b1ab5028726 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:50.298981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15162ce-3100-40db-a9ec-5b1ab5028726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:50.344199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:50.389089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f15162ce-3100-40db-a9ec-5b1ab5028726 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:50.390466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f15162ce-3100-40db-a9ec-5b1ab5028726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:50.522980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 684/1370] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:50.858233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48040c2a-d182-41af-af7a-14f1f429f56a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:50.858525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48040c2a-d182-41af-af7a-14f1f429f56a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:50.858807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48040c2a-d182-41af-af7a-14f1f429f56a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:50.860442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 687/1371] 10.222.0.72 () {54 vars in 963 bytes} [Mon Sep 26 17:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:50.866400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b664173-6140-4d8b-affb-96876fc9ea1f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:50.866694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b664173-6140-4d8b-affb-96876fc9ea1f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:50.867006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b664173-6140-4d8b-affb-96876fc9ea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:50.949465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4b664173-6140-4d8b-affb-96876fc9ea1f None None] MFA Rules not processed for user `39bde61b0b7c4cf3950ef63248a3b0da`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:50.950708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4b664173-6140-4d8b-affb-96876fc9ea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:51.007310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:51.048376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 685/1372] 10.222.0.72 () {58 vars in 1060 bytes} [Mon Sep 26 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:51.056472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48040c2a-d182-41af-af7a-14f1f429f56a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:51.058606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48040c2a-d182-41af-af7a-14f1f429f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:51.116767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:51.177279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48040c2a-d182-41af-af7a-14f1f429f56a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:51.266382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:51.267206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:51.267590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:51.267981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:51.271091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:51.404571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:51.423561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:51.426014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b447436-3a9e-4abb-8454-28ec3b918bf2 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:51.475817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 688/1373] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:51:52.484652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a06c-1a1e-426a-9dd2-d4a87acff031 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:52.485219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a06c-1a1e-426a-9dd2-d4a87acff031 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:52.485219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685a06c-1a1e-426a-9dd2-d4a87acff031 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:52.486585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 686/1374] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:52.492760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913696b-fa8c-457b-b608-0d288ef65c8b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:52.493182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913696b-fa8c-457b-b608-0d288ef65c8b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:52.493462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913696b-fa8c-457b-b608-0d288ef65c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:52.559871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e913696b-fa8c-457b-b608-0d288ef65c8b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:52.561179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e913696b-fa8c-457b-b608-0d288ef65c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:52.659707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 689/1375] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:52.955224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685a06c-1a1e-426a-9dd2-d4a87acff031 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:52.957575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2685a06c-1a1e-426a-9dd2-d4a87acff031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:53.123214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685a06c-1a1e-426a-9dd2-d4a87acff031 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:53.177879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:53.243320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:53.244098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:53.244388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:53.244660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:53.246310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:53.340001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:53.341601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d6107-92da-46b4-b87c-562561028f8b service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:53.361726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 687/1376] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:55.614594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a458f7-d612-4598-b780-2dc6db082ea7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:55.615214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a458f7-d612-4598-b780-2dc6db082ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:55.615214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a458f7-d612-4598-b780-2dc6db082ea7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:55.616802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 690/1377] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:55.622241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602614-b1e4-482a-8bd2-ab05fcc558b7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:55.622679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602614-b1e4-482a-8bd2-ab05fcc558b7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:55.622891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c602614-b1e4-482a-8bd2-ab05fcc558b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:55.703323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-0c602614-b1e4-482a-8bd2-ab05fcc558b7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:55.704540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0c602614-b1e4-482a-8bd2-ab05fcc558b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:55.712218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:55.816913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 688/1378] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:51:56.110443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a458f7-d612-4598-b780-2dc6db082ea7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:56.112024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-25a458f7-d612-4598-b780-2dc6db082ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:56.177907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:56.269368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a458f7-d612-4598-b780-2dc6db082ea7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:56.387477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:56.388093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:56.388378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:56.388650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:56.390251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:56.412126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:56.488421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:56.490045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70ccf6-72f3-42d9-8e75-7e727cf2aa33 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:56.509562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 691/1379] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:51:58.783017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7f56-e851-426c-aa42-a87c47108c30 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:58.784178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7f56-e851-426c-aa42-a87c47108c30 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:58.784178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7f56-e851-426c-aa42-a87c47108c30 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:58.785670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 689/1380] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:51:58.790744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b3c7f-26c6-4f1b-9cad-f204a74603bb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:58.791195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b3c7f-26c6-4f1b-9cad-f204a74603bb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:58.791369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b3c7f-26c6-4f1b-9cad-f204a74603bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:58.860865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ad9b3c7f-26c6-4f1b-9cad-f204a74603bb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:51:58.862121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ad9b3c7f-26c6-4f1b-9cad-f204a74603bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:58.971322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:58.975251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 692/1381] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:51:59.274429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d7f56-e851-426c-aa42-a87c47108c30 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:51:59.276402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ff0d7f56-e851-426c-aa42-a87c47108c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:59.436349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:59.452344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d7f56-e851-426c-aa42-a87c47108c30 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:51:59.600688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:51:59.601626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:51:59.601994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:51:59.602354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:51:59.604556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:51:59.712118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:51:59.714027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7fa48-8af6-4b8f-840a-609d374d4c2f service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:51:59.727434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:51:59.739492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 690/1382] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:01.981800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ce7a-0f8b-4504-acb3-b15fb8c56849 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:01.982755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ce7a-0f8b-4504-acb3-b15fb8c56849 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:01.982755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ce7a-0f8b-4504-acb3-b15fb8c56849 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:01.984297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 693/1383] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:01.992122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ecbc2-84ed-4ef1-9147-3238022a0bde None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:01.993739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ecbc2-84ed-4ef1-9147-3238022a0bde None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:01.994459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ecbc2-84ed-4ef1-9147-3238022a0bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:02.090476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-565ecbc2-84ed-4ef1-9147-3238022a0bde None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:02.091761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-565ecbc2-84ed-4ef1-9147-3238022a0bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:02.205342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 691/1384] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:02.504148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ce7a-0f8b-4504-acb3-b15fb8c56849 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:02.505800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8e91ce7a-0f8b-4504-acb3-b15fb8c56849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:02.697264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ce7a-0f8b-4504-acb3-b15fb8c56849 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:02.775276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:02.832554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:02.833158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:02.833440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:02.833708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:02.835446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:02.932064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:02.934433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed27e93-4343-457b-9ec6-3704ad01d9cd service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:02.959222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 694/1385] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:05.141251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee8c2-8f76-4f84-99cd-4c63cf0726d0 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:05.141863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee8c2-8f76-4f84-99cd-4c63cf0726d0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:05.141863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee8c2-8f76-4f84-99cd-4c63cf0726d0 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:05.143367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 692/1386] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:05.149015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47db17-3294-4de3-bc33-9e584685da00 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:05.149447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47db17-3294-4de3-bc33-9e584685da00 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:05.149552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47db17-3294-4de3-bc33-9e584685da00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:05.216920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bf47db17-3294-4de3-bc33-9e584685da00 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:05.218121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bf47db17-3294-4de3-bc33-9e584685da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:05.254232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:05.319572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 695/1387] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:05.617355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee8c2-8f76-4f84-99cd-4c63cf0726d0 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:05.619229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9b3ee8c2-8f76-4f84-99cd-4c63cf0726d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:05.714509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:05.793587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee8c2-8f76-4f84-99cd-4c63cf0726d0 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:05.912744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:05.913357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:05.913632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:05.913896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:05.915507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:05.954284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:06.010850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:06.012498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9048f-41e8-4eb2-bff2-b280f1c8e5c3 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:06.036086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 693/1388] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:07.004224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068128-d91a-475a-9a61-454edbb588e3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:07.004719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068128-d91a-475a-9a61-454edbb588e3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:07.005006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068128-d91a-475a-9a61-454edbb588e3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:07.006409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 696/1389] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:07.011878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c620aaa-f63b-4cab-a2c7-8765b8b4eea2 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:07.012291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c620aaa-f63b-4cab-a2c7-8765b8b4eea2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:07.012465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c620aaa-f63b-4cab-a2c7-8765b8b4eea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:07.081267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4c620aaa-f63b-4cab-a2c7-8765b8b4eea2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:07.082667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4c620aaa-f63b-4cab-a2c7-8765b8b4eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:07.180795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 694/1390] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:07.483562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28172a-8ad9-4019-b757-b7d7da810aa1 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:07.483844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28172a-8ad9-4019-b757-b7d7da810aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:07.484120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28172a-8ad9-4019-b757-b7d7da810aa1 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:07.485461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 697/1391] 10.222.0.72 () {54 vars in 962 bytes} [Mon Sep 26 17:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:52:07.491535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e401aa4-18fb-4851-a0de-daa1ea4aed46 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:07.491819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e401aa4-18fb-4851-a0de-daa1ea4aed46 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:07.492114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e401aa4-18fb-4851-a0de-daa1ea4aed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:07.508231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:07.569048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3e401aa4-18fb-4851-a0de-daa1ea4aed46 None None] MFA Rules not processed for user `10f80305dfe04303a9a9387535ed8ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:07.570267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3e401aa4-18fb-4851-a0de-daa1ea4aed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:07.646346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 695/1392] 10.222.0.72 () {58 vars in 1059 bytes} [Mon Sep 26 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:07.655227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28172a-8ad9-4019-b757-b7d7da810aa1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:07.656861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fa28172a-8ad9-4019-b757-b7d7da810aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:07.765967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28172a-8ad9-4019-b757-b7d7da810aa1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:07.833692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:07.851423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:07.852030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:07.852305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:07.852575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:07.854153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:07.967927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:07.969399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e4e5-8faf-4bbf-bfc7-a232eb43f64e service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:07.989860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 698/1393] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:08.276495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3685a2-631b-4ff5-acd6-2e860282a83d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:08.277065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3685a2-631b-4ff5-acd6-2e860282a83d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:08.277065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3685a2-631b-4ff5-acd6-2e860282a83d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:08.278523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 696/1394] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:08.284233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1261f2-5667-4413-b067-631fe032046e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:08.284655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1261f2-5667-4413-b067-631fe032046e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:08.284785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1261f2-5667-4413-b067-631fe032046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:08.366141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1c1261f2-5667-4413-b067-631fe032046e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:08.367551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1c1261f2-5667-4413-b067-631fe032046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:08.420943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:08.479354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 699/1395] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:08.748526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d329d-63f3-4a20-b091-c3fb239fbff6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:08.748894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d329d-63f3-4a20-b091-c3fb239fbff6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:08.749243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d329d-63f3-4a20-b091-c3fb239fbff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:08.783468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1261f2-5667-4413-b067-631fe032046e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:08.785256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1c1261f2-5667-4413-b067-631fe032046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:08.819110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-1a2d329d-63f3-4a20-b091-c3fb239fbff6 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:08.820331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1a2d329d-63f3-4a20-b091-c3fb239fbff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:08.877013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:08.904141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1261f2-5667-4413-b067-631fe032046e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:08.911123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 697/1396] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 17:52:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:08.955052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b1d64-271a-4f8d-992d-617a361f56c8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:08.955341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b1d64-271a-4f8d-992d-617a361f56c8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:08.955709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b1d64-271a-4f8d-992d-617a361f56c8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:08.957673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 698/1397] 10.222.0.72 () {54 vars in 943 bytes} [Mon Sep 26 17:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:52:08.963768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80da44c-b707-4dae-9199-51ec7279614e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:08.964259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80da44c-b707-4dae-9199-51ec7279614e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:08.964543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80da44c-b707-4dae-9199-51ec7279614e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:08.965018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:09.027186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:09.027831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:09.028116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:09.028797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:09.029988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:09.048993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a80da44c-b707-4dae-9199-51ec7279614e None None] MFA Rules not processed for user `d2ba853553814d49bf3c3a92c29fd27f`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:09.050415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a80da44c-b707-4dae-9199-51ec7279614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:09.125574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:09.127301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:09.147083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 700/1398] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:52:09.159499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 699/1399] 10.222.0.72 () {58 vars in 1040 bytes} [Mon Sep 26 17:52:08 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:52:09.168851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:09.170069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:09.240187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:09.270732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc4572-cfc0-4f26-a3a1-0625889575f9 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:09.389201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:09.389845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:09.390079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:09.390352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:09.392049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:09.476652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:09.478257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187ac7f-37c6-47b3-856c-63d36b6eb3d1 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:09.486377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 701/1400] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 17:52:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 17:52:11.429540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8202448-a032-434e-9ef9-d903a014597d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:11.430095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8202448-a032-434e-9ef9-d903a014597d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:11.430095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8202448-a032-434e-9ef9-d903a014597d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:11.432829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 700/1401] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:11.438226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39df8a-fa9c-4986-b941-4ceb381de1b6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:11.438688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39df8a-fa9c-4986-b941-4ceb381de1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:11.438795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39df8a-fa9c-4986-b941-4ceb381de1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:11.455552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:11.517030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2e39df8a-fa9c-4986-b941-4ceb381de1b6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:11.518958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2e39df8a-fa9c-4986-b941-4ceb381de1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:11.679501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 702/1402] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:11.973858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8202448-a032-434e-9ef9-d903a014597d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:11.975412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b8202448-a032-434e-9ef9-d903a014597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:12.013245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:12.157832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8202448-a032-434e-9ef9-d903a014597d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:12.276196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:12.288335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:12.288951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:12.289232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:12.289512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:12.291161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:12.388670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:12.390365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3fdf3-1175-47c8-b9fa-ac4889ae6605 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:12.414127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 701/1403] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:14.698356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c6a4d-600c-4657-9250-0718768dda83 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:14.698977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c6a4d-600c-4657-9250-0718768dda83 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:14.699088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c6a4d-600c-4657-9250-0718768dda83 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:14.702871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 703/1404] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:14.706728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3174e8-ef6d-48e0-9304-5f24e58b56d6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:14.707226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3174e8-ef6d-48e0-9304-5f24e58b56d6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:14.707435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3174e8-ef6d-48e0-9304-5f24e58b56d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:14.786329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-eb3174e8-ef6d-48e0-9304-5f24e58b56d6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:14.787608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-eb3174e8-ef6d-48e0-9304-5f24e58b56d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:14.855532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:14.903663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 702/1405] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:15.201920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c6a4d-600c-4657-9250-0718768dda83 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:15.203490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f88c6a4d-600c-4657-9250-0718768dda83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:15.298853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:15.373503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c6a4d-600c-4657-9250-0718768dda83 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:15.504580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:15.505193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:15.505474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:15.505748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:15.507406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:15.569483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:15.611481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:15.616569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff9b2e-5ca4-4c5f-958f-c123c17d311d service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:15.639013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 704/1406] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:17.959994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043531a-2876-47a4-90a1-f96df6798ffa None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:17.961066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043531a-2876-47a4-90a1-f96df6798ffa None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:17.961589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043531a-2876-47a4-90a1-f96df6798ffa None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:17.963079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 703/1407] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:17.971056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f03b8-e26f-42cf-b673-5cafc5f3c2df None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:17.972249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f03b8-e26f-42cf-b673-5cafc5f3c2df None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:17.972499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f03b8-e26f-42cf-b673-5cafc5f3c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:18.049045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-899f03b8-e26f-42cf-b673-5cafc5f3c2df None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:18.051020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-899f03b8-e26f-42cf-b673-5cafc5f3c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:18.155030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 705/1408] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:18.450678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043531a-2876-47a4-90a1-f96df6798ffa None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:18.452237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f043531a-2876-47a4-90a1-f96df6798ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:18.618089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043531a-2876-47a4-90a1-f96df6798ffa None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:18.680228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:18.750301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:18.750944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:18.751235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:18.751499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:18.753752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:18.856923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:18.858544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e0418-4963-45df-ab77-f01ba8480a0e service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:18.878185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 704/1409] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:28.153113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=87 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:28.875174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=76 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:32.495120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd65572-2f96-405b-a382-7163b541c6f5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:32.495726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd65572-2f96-405b-a382-7163b541c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:32.495726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd65572-2f96-405b-a382-7163b541c6f5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:32.497024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 706/1410] 10.222.0.72 () {54 vars in 918 bytes} [Mon Sep 26 17:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:32.502252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96855ceb-5c64-4e4f-bc52-95d1f78bcf5d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:32.502869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96855ceb-5c64-4e4f-bc52-95d1f78bcf5d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:32.502869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96855ceb-5c64-4e4f-bc52-95d1f78bcf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:32.585029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-96855ceb-5c64-4e4f-bc52-95d1f78bcf5d None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:32.586232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-96855ceb-5c64-4e4f-bc52-95d1f78bcf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:32.692744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 705/1411] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Sep 26 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:32.707641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0dc9e-2063-4873-a8ed-3c5c0e832473 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:32.707922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0dc9e-2063-4873-a8ed-3c5c0e832473 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:32.708194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0dc9e-2063-4873-a8ed-3c5c0e832473 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:32.709618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 707/1412] 10.222.0.72 () {54 vars in 918 bytes} [Mon Sep 26 17:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:32.714433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69552d7c-98dc-4541-8e04-f98f69d377ba None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:32.714720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69552d7c-98dc-4541-8e04-f98f69d377ba None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:32.715025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69552d7c-98dc-4541-8e04-f98f69d377ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:32.789400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:32.792396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-69552d7c-98dc-4541-8e04-f98f69d377ba None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:32.793683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-69552d7c-98dc-4541-8e04-f98f69d377ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:32.915445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 706/1413] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Sep 26 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:52:35.562914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c08544-c770-4a25-82d7-7b8916d5b268 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:35.563486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c08544-c770-4a25-82d7-7b8916d5b268 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:35.563486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c08544-c770-4a25-82d7-7b8916d5b268 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:35.564843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:35.570132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53595e-2c8b-4ef7-b037-e611898c8522 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:35.570549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53595e-2c8b-4ef7-b037-e611898c8522 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:35.570671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53595e-2c8b-4ef7-b037-e611898c8522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:35.637358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-db53595e-2c8b-4ef7-b037-e611898c8522 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:35.638560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-db53595e-2c8b-4ef7-b037-e611898c8522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:35.693364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:35.736749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:36.030425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c08544-c770-4a25-82d7-7b8916d5b268 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:36.032323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e9c08544-c770-4a25-82d7-7b8916d5b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:36.184518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c08544-c770-4a25-82d7-7b8916d5b268 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:36.257741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:36.297387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:36.297983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:36.298309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:36.298562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:36.300713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:36.390575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:36.392225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8b47a-a968-440f-88d4-236364d36152 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:36.410109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 709/1416] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:37.561342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad9518-c9fb-4ff6-94da-91551325206d None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:37.561911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad9518-c9fb-4ff6-94da-91551325206d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:37.561911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad9518-c9fb-4ff6-94da-91551325206d None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:37.563300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:37.569886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eec1e4-4673-4992-8518-cf3b8ed34a4d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:37.570329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eec1e4-4673-4992-8518-cf3b8ed34a4d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:37.570467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eec1e4-4673-4992-8518-cf3b8ed34a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:37.648794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-11eec1e4-4673-4992-8518-cf3b8ed34a4d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:37.650004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-11eec1e4-4673-4992-8518-cf3b8ed34a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:37.677553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:37.747465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:37.899600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad9518-c9fb-4ff6-94da-91551325206d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:37.901053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4fad9518-c9fb-4ff6-94da-91551325206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:38.052866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad9518-c9fb-4ff6-94da-91551325206d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:38.093922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:38.194605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:38.195167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:38.195451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:38.195780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:38.197634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:38.306489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:38.309076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe24dbe-59f2-4fe3-bbcc-1a4636e56644 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:38.332601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 709/1419] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:39.526538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315c1d-91fe-41f5-a7d1-86f7b9df7069 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:39.527134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315c1d-91fe-41f5-a7d1-86f7b9df7069 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:39.527134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315c1d-91fe-41f5-a7d1-86f7b9df7069 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:39.528479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:39.534621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75a898-dffd-48b7-91f6-84648be1eb0e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:39.535173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75a898-dffd-48b7-91f6-84648be1eb0e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:39.535173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75a898-dffd-48b7-91f6-84648be1eb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:39.589330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:39.611560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ac75a898-dffd-48b7-91f6-84648be1eb0e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:39.613406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ac75a898-dffd-48b7-91f6-84648be1eb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:39.709889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:39.863580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315c1d-91fe-41f5-a7d1-86f7b9df7069 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:39.865097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c0315c1d-91fe-41f5-a7d1-86f7b9df7069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:40.014799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315c1d-91fe-41f5-a7d1-86f7b9df7069 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:40.027003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:40.129001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:40.129503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:40.129782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:40.130052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:40.131694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:40.223786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:40.225374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8d3f3-ac8d-4a4e-8b8f-7ae44eae475b service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:40.243561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 712/1422] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:41.362195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14c8ad-8a5c-4a55-ac48-106577e6dd16 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:41.362751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14c8ad-8a5c-4a55-ac48-106577e6dd16 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:41.362751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14c8ad-8a5c-4a55-ac48-106577e6dd16 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:41.364187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:41.370535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d98736-4694-42cf-aea8-ee5b23e04c80 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:41.371064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d98736-4694-42cf-aea8-ee5b23e04c80 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:41.371191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d98736-4694-42cf-aea8-ee5b23e04c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:41.390772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:41.448134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-13d98736-4694-42cf-aea8-ee5b23e04c80 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:41.449350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-13d98736-4694-42cf-aea8-ee5b23e04c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:41.546447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:41.698443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14c8ad-8a5c-4a55-ac48-106577e6dd16 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:41.699897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0e14c8ad-8a5c-4a55-ac48-106577e6dd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:41.809350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:41.846120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14c8ad-8a5c-4a55-ac48-106577e6dd16 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:41.958903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:41.959518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:41.959803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:41.960079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:41.961689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:42.040919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:42.056199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:42.057798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21a3c3-c93f-487e-bce1-a438a51201b7 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:42.076072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 712/1425] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:43.194511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead268a7-52e4-4778-9921-92e197dd02da None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:43.195210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead268a7-52e4-4778-9921-92e197dd02da None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:43.195210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead268a7-52e4-4778-9921-92e197dd02da None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:43.196722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:43.202889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d793-d1f1-4055-87a9-8cae572e17b6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:43.203356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d793-d1f1-4055-87a9-8cae572e17b6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:43.203462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9d793-d1f1-4055-87a9-8cae572e17b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:43.279375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-39d9d793-d1f1-4055-87a9-8cae572e17b6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:43.280496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-39d9d793-d1f1-4055-87a9-8cae572e17b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:43.378206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:43.528841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead268a7-52e4-4778-9921-92e197dd02da None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:43.530532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ead268a7-52e4-4778-9921-92e197dd02da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:43.617325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:43.679975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead268a7-52e4-4778-9921-92e197dd02da None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:43.809241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:43.809765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:43.810041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:43.810334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:43.811947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:43.861007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:43.903902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:43.905511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5275d32-a279-4aa8-87cc-83d92791613c service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:43.923480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 715/1428] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:45.001147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c14472-3a32-43ce-aed4-db3c104b4a35 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:45.001655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c14472-3a32-43ce-aed4-db3c104b4a35 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:45.001765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c14472-3a32-43ce-aed4-db3c104b4a35 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:45.003095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:45.009999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0194a-a238-44fa-a946-a25e45fa3229 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:45.010471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0194a-a238-44fa-a946-a25e45fa3229 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:45.010570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0194a-a238-44fa-a946-a25e45fa3229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:45.090249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-aff0194a-a238-44fa-a946-a25e45fa3229 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:45.091492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aff0194a-a238-44fa-a946-a25e45fa3229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:45.189354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:45.343449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c14472-3a32-43ce-aed4-db3c104b4a35 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:45.344885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-91c14472-3a32-43ce-aed4-db3c104b4a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:45.401234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:45.500190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c14472-3a32-43ce-aed4-db3c104b4a35 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:45.613589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:45.614124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:45.614400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:45.614667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:45.616343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:45.623263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:45.707586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:45.709679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fd5fe-91e2-4055-b53a-bd2162752471 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:45.728346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 715/1431] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:46.810254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897b666-2071-4fe1-8494-c110a6101553 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:46.810865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897b666-2071-4fe1-8494-c110a6101553 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:46.810865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897b666-2071-4fe1-8494-c110a6101553 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:46.812269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:46.819373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66752ed0-bfd3-41f8-824a-7579c2540d10 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:46.819855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66752ed0-bfd3-41f8-824a-7579c2540d10 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:46.819964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66752ed0-bfd3-41f8-824a-7579c2540d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:46.896469 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-66752ed0-bfd3-41f8-824a-7579c2540d10 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:46.897676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-66752ed0-bfd3-41f8-824a-7579c2540d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:46.965458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:46.995188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:47.153441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897b666-2071-4fe1-8494-c110a6101553 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:47.154892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3897b666-2071-4fe1-8494-c110a6101553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:47.172811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:47.301025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897b666-2071-4fe1-8494-c110a6101553 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:47.395009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:47.420858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:47.421383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:47.421659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:47.421929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:47.423536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:47.511868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:47.513456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286635f0-72c4-4dd9-ac7c-54b6df116fcd service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:47.530979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 718/1434] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:48.590737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57822f0-a2a1-4ab2-8a43-caad71735140 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:48.591343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57822f0-a2a1-4ab2-8a43-caad71735140 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:48.591343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57822f0-a2a1-4ab2-8a43-caad71735140 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:48.592665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:52:48.599031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e2c89-8677-4571-b7cd-28ee1bfd4112 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:48.599511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e2c89-8677-4571-b7cd-28ee1bfd4112 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:48.599618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e2c89-8677-4571-b7cd-28ee1bfd4112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:48.676874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ba6e2c89-8677-4571-b7cd-28ee1bfd4112 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:48.678087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba6e2c89-8677-4571-b7cd-28ee1bfd4112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:48.736492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:48.792315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:48.945586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57822f0-a2a1-4ab2-8a43-caad71735140 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:48.947334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c57822f0-a2a1-4ab2-8a43-caad71735140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:49.107016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57822f0-a2a1-4ab2-8a43-caad71735140 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:49.163253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:49.222364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:49.223034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:49.223345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:49.223613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:49.225269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:49.314122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:49.316592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aff831-8d9e-47ed-b899-59fa3dc7c9d9 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:49.335693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 718/1437] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:50.390976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43186-8a58-4aff-93fc-2bcedf4b745a None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:50.391527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43186-8a58-4aff-93fc-2bcedf4b745a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:50.391527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43186-8a58-4aff-93fc-2bcedf4b745a None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:50.392796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:50.399675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1016a-07d3-4dca-bed5-c4d327a2a83c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:50.400130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1016a-07d3-4dca-bed5-c4d327a2a83c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:50.400253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1016a-07d3-4dca-bed5-c4d327a2a83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:50.477962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c2e1016a-07d3-4dca-bed5-c4d327a2a83c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:50.479211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c2e1016a-07d3-4dca-bed5-c4d327a2a83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:50.481205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:50.576357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:50.727081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a43186-8a58-4aff-93fc-2bcedf4b745a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:50.728594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-49a43186-8a58-4aff-93fc-2bcedf4b745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:50.874798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a43186-8a58-4aff-93fc-2bcedf4b745a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:50.902374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:50.989232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:50.989841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:50.990124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:50.990397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:50.992035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:51.085128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:51.086717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197b8a7-2bad-4c20-a5c4-f95260642bc6 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:51.104539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 721/1440] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:53.170168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a75df-4b38-43fb-a5b7-b08ac55700b1 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:53.170734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a75df-4b38-43fb-a5b7-b08ac55700b1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:53.170734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a75df-4b38-43fb-a5b7-b08ac55700b1 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:53.172152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:53.178125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98818c-47a3-4e85-b232-b6e7644fd92b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:53.178557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98818c-47a3-4e85-b232-b6e7644fd92b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:53.178677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98818c-47a3-4e85-b232-b6e7644fd92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:53.219603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:53.246406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5c98818c-47a3-4e85-b232-b6e7644fd92b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:53.247541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c98818c-47a3-4e85-b232-b6e7644fd92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:53.346210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:52:53.636346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a75df-4b38-43fb-a5b7-b08ac55700b1 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:53.637878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-386a75df-4b38-43fb-a5b7-b08ac55700b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:53.742297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a75df-4b38-43fb-a5b7-b08ac55700b1 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:53.774525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:53.817680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:53.818290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:53.818574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:53.818881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:53.820495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:53.930768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:53.932153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2309-e64b-4905-9707-ab26bf528caf service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:53.952345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 721/1443] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:56.261217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239701d-792c-45e7-92c6-21f824a6729d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:56.261750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239701d-792c-45e7-92c6-21f824a6729d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:56.261750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239701d-792c-45e7-92c6-21f824a6729d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:56.263066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:56.268660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bbdc-0862-4f9b-b455-16578a3ea154 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:56.269097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bbdc-0862-4f9b-b455-16578a3ea154 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:56.269310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bbdc-0862-4f9b-b455-16578a3ea154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:56.336072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6a51bbdc-0862-4f9b-b455-16578a3ea154 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:56.337296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6a51bbdc-0862-4f9b-b455-16578a3ea154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:56.356241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:56.441256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:52:56.736093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239701d-792c-45e7-92c6-21f824a6729d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:52:56.737772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a239701d-792c-45e7-92c6-21f824a6729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:56.856629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239701d-792c-45e7-92c6-21f824a6729d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:52:56.873139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:56.941186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:52:56.941788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:56.942072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:56.942345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:56.944008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:57.056371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:52:57.057829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189c978-867d-4110-8cab-2523eeb91b87 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:52:57.076252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 724/1446] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:52:59.572933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb1e2-5d29-4dde-99d8-784c847dbf32 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:59.573491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb1e2-5d29-4dde-99d8-784c847dbf32 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:59.573604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfb1e2-5d29-4dde-99d8-784c847dbf32 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:59.574941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:52:59.580262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee00d9b-ea96-4d52-8fe9-5498cb171046 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:52:59.580675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee00d9b-ea96-4d52-8fe9-5498cb171046 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:52:59.580817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee00d9b-ea96-4d52-8fe9-5498cb171046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:52:59.641460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:52:59.660566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5ee00d9b-ea96-4d52-8fe9-5498cb171046 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:52:59.661777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5ee00d9b-ea96-4d52-8fe9-5498cb171046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:52:59.758708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:00.048164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfb1e2-5d29-4dde-99d8-784c847dbf32 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:00.049821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0abfb1e2-5d29-4dde-99d8-784c847dbf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:00.155811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfb1e2-5d29-4dde-99d8-784c847dbf32 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:00.196461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:00.233342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:00.233864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:00.234147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:00.234421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:00.236069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:00.352421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:00.353855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1205d-d58c-49e0-b5d7-60ed664a100a service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:00.373560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 724/1449] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:02.745407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7cb00-3c4b-4a6b-a288-4704be1123db None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:02.745978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7cb00-3c4b-4a6b-a288-4704be1123db None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:02.745978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7cb00-3c4b-4a6b-a288-4704be1123db None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:02.747362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:02.752744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52998a-8323-4346-a0be-dc0ca6076922 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:02.754291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52998a-8323-4346-a0be-dc0ca6076922 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:02.754291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52998a-8323-4346-a0be-dc0ca6076922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:02.820337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-0a52998a-8323-4346-a0be-dc0ca6076922 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:02.821798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0a52998a-8323-4346-a0be-dc0ca6076922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:02.842378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:02.919860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:03.217902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7cb00-3c4b-4a6b-a288-4704be1123db None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:03.219470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-72e7cb00-3c4b-4a6b-a288-4704be1123db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:03.325976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7cb00-3c4b-4a6b-a288-4704be1123db None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:03.463078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:03.516917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:03.517527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:03.517812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:03.518080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:03.520420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:03.628680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:03.630043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7c9ee-dd16-4ce7-b9c1-dba8d08bcd26 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:03.648710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 727/1452] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:05.898281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e179c2-78c5-4c59-b2d5-27250a166f15 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:05.898865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e179c2-78c5-4c59-b2d5-27250a166f15 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:05.898865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e179c2-78c5-4c59-b2d5-27250a166f15 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:05.900227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:05.905477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d51f-b621-4810-a265-73ca570d206b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:05.905883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d51f-b621-4810-a265-73ca570d206b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:05.906025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87d51f-b621-4810-a265-73ca570d206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:05.973691 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:05.977396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ac87d51f-b621-4810-a265-73ca570d206b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:05.978822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ac87d51f-b621-4810-a265-73ca570d206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:06.077750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:06.369624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e179c2-78c5-4c59-b2d5-27250a166f15 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:06.371214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d9e179c2-78c5-4c59-b2d5-27250a166f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:06.477240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e179c2-78c5-4c59-b2d5-27250a166f15 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:06.527507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:06.552664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:06.553271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:06.553554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:06.553830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:06.555461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:06.665845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:06.667241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4379b5-f39a-41c3-8d94-0a5587a02fda service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:06.685388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 727/1455] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:08.962202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752002-10fe-4b58-8828-6b43cebfb6f7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:08.962765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752002-10fe-4b58-8828-6b43cebfb6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:08.962765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3752002-10fe-4b58-8828-6b43cebfb6f7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:08.964148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:08.969721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601769-6442-4542-beb7-d4a0e5981db1 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:08.970132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601769-6442-4542-beb7-d4a0e5981db1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:08.970362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a601769-6442-4542-beb7-d4a0e5981db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:09.039153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7a601769-6442-4542-beb7-d4a0e5981db1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:09.040370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7a601769-6442-4542-beb7-d4a0e5981db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:09.069791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:09.140684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:09.427631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3752002-10fe-4b58-8828-6b43cebfb6f7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:09.429211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d3752002-10fe-4b58-8828-6b43cebfb6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:09.533028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3752002-10fe-4b58-8828-6b43cebfb6f7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:09.561656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:09.607029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:09.607916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:09.608193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:09.608461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:09.610048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:09.719918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:09.721951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dca485-f6b2-445e-b387-e977225232b0 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:09.740726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 730/1458] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:11.983712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34357-23e1-4f11-8f12-46d521fbbd2a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:11.984098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34357-23e1-4f11-8f12-46d521fbbd2a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:11.984273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d34357-23e1-4f11-8f12-46d521fbbd2a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:11.985647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:11.991677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca0716-db1d-4983-9e5d-0dd53a2dc0c5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:11.992094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca0716-db1d-4983-9e5d-0dd53a2dc0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:11.992230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca0716-db1d-4983-9e5d-0dd53a2dc0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:12.058946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4fca0716-db1d-4983-9e5d-0dd53a2dc0c5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:12.060155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4fca0716-db1d-4983-9e5d-0dd53a2dc0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:12.068475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:12.156521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:12.440169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34357-23e1-4f11-8f12-46d521fbbd2a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:12.442415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-20d34357-23e1-4f11-8f12-46d521fbbd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:12.547441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d34357-23e1-4f11-8f12-46d521fbbd2a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:12.603719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:12.624360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:12.624990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:12.625272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:12.625545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:12.627222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:12.736360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:12.738407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a9e47-4ca4-436f-8fae-8b3524f630b2 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:12.755907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 730/1461] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:15.011479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51fcb-605d-41aa-aa1f-377453828d65 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:15.012030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51fcb-605d-41aa-aa1f-377453828d65 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:15.012030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51fcb-605d-41aa-aa1f-377453828d65 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:15.013410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:15.019782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1264457-361d-478e-9faa-8a0f99e28652 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:15.020230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1264457-361d-478e-9faa-8a0f99e28652 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:15.020344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1264457-361d-478e-9faa-8a0f99e28652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:15.086895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a1264457-361d-478e-9faa-8a0f99e28652 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:15.088110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a1264457-361d-478e-9faa-8a0f99e28652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:15.126757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:15.186265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:15.477902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c51fcb-605d-41aa-aa1f-377453828d65 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:15.479557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-55c51fcb-605d-41aa-aa1f-377453828d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:15.583802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c51fcb-605d-41aa-aa1f-377453828d65 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:15.623297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:15.658639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:15.660038 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:15.660163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:15.660426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:15.662028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:15.770700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:15.772088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9838a2-e532-4931-b6ef-1057c7b201fe service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:15.790713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 733/1464] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:17.988171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2d30-dcf3-441c-a37f-9e045026232e None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:17.988642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2d30-dcf3-441c-a37f-9e045026232e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:17.988786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2d30-dcf3-441c-a37f-9e045026232e None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:17.990105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:17.995370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18989660-1ed3-4676-9e21-d2620cbf8f3d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:17.995796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18989660-1ed3-4676-9e21-d2620cbf8f3d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:17.995900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18989660-1ed3-4676-9e21-d2620cbf8f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:18.061394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-18989660-1ed3-4676-9e21-d2620cbf8f3d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:18.062597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-18989660-1ed3-4676-9e21-d2620cbf8f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:18.082207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:18.170743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:18.455525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2d30-dcf3-441c-a37f-9e045026232e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:18.457075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-90fe2d30-dcf3-441c-a37f-9e045026232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:18.561864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2d30-dcf3-441c-a37f-9e045026232e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:18.633314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:18.640824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:18.641431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:18.641713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:18.641986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:18.643811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:18.753208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:18.754640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796ec73-a842-4209-aaae-f92315ad9a4d service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:18.772533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 733/1467] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:21.027209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241acbf-147e-4e50-8aac-dd311967f9a6 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:21.027756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241acbf-147e-4e50-8aac-dd311967f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:21.027860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241acbf-147e-4e50-8aac-dd311967f9a6 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:21.029163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:21.034767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9099f-fcd4-4431-a326-708a85ee8dc3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:21.035193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9099f-fcd4-4431-a326-708a85ee8dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:21.035349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9099f-fcd4-4431-a326-708a85ee8dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:21.102676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-31e9099f-fcd4-4431-a326-708a85ee8dc3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:21.103931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-31e9099f-fcd4-4431-a326-708a85ee8dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:21.152677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:21.202599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:21.488882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241acbf-147e-4e50-8aac-dd311967f9a6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:21.490431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2241acbf-147e-4e50-8aac-dd311967f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:21.593773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241acbf-147e-4e50-8aac-dd311967f9a6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:21.644812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:21.670414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:21.671054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:21.671336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:21.671608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:21.673204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:21.784743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:21.786095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d610831-6fbc-4b7e-9b5f-20226552de22 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:21.804308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 736/1470] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:31.201082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=22 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:31.801992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=66 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:42.134006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e76f-a978-4f6b-b189-86f2d733a13b None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:42.134637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e76f-a978-4f6b-b189-86f2d733a13b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:42.134637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e76f-a978-4f6b-b189-86f2d733a13b None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:42.136028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 735/1471] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:42.141431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d4a7-d787-4984-81d1-45cbabac4c7c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:42.141839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d4a7-d787-4984-81d1-45cbabac4c7c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:42.141990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d4a7-d787-4984-81d1-45cbabac4c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:42.209107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6db7d4a7-d787-4984-81d1-45cbabac4c7c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:42.210330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6db7d4a7-d787-4984-81d1-45cbabac4c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:42.306967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 737/1472] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:42.612050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca056a-b774-477f-9a90-1f2780fa0fd3 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:42.612336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca056a-b774-477f-9a90-1f2780fa0fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:42.612782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca056a-b774-477f-9a90-1f2780fa0fd3 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:42.614135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 736/1473] 10.222.0.72 () {54 vars in 962 bytes} [Mon Sep 26 17:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:42.619198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81292d-9759-48bc-bd36-0f3bb22ab218 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:42.619568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81292d-9759-48bc-bd36-0f3bb22ab218 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:42.619759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81292d-9759-48bc-bd36-0f3bb22ab218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:42.697958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ce81292d-9759-48bc-bd36-0f3bb22ab218 None None] MFA Rules not processed for user `10f80305dfe04303a9a9387535ed8ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:42.699557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ce81292d-9759-48bc-bd36-0f3bb22ab218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:42.724859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:42.790238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 738/1474] 10.222.0.72 () {58 vars in 1059 bytes} [Mon Sep 26 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:42.797934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca056a-b774-477f-9a90-1f2780fa0fd3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:42.800258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-35ca056a-b774-477f-9a90-1f2780fa0fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:43.024133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca056a-b774-477f-9a90-1f2780fa0fd3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:43.123684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:43.124318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:43.124597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:43.124870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:43.126482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:43.180612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:43.236582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:43.237946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf140c-5957-4c70-b7ec-059eb2c34996 service glance] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:43.255821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 737/1475] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:45.506301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eacba2-df68-44d7-b3e1-3fb297eea270 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:45.506958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eacba2-df68-44d7-b3e1-3fb297eea270 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:45.506958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eacba2-df68-44d7-b3e1-3fb297eea270 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:45.508239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 739/1476] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:45.513385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e5607-ccab-415a-9c76-39709237f4a3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:45.513815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e5607-ccab-415a-9c76-39709237f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:45.513922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e5607-ccab-415a-9c76-39709237f4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:45.580026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-011e5607-ccab-415a-9c76-39709237f4a3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:45.581832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-011e5607-ccab-415a-9c76-39709237f4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:45.679188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 738/1477] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:45.951402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eacba2-df68-44d7-b3e1-3fb297eea270 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:45.952930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e8eacba2-df68-44d7-b3e1-3fb297eea270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:46.077891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eacba2-df68-44d7-b3e1-3fb297eea270 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:46.129530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:46.176178 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:46.176789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:46.177074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:46.177352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:46.177988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:46.179822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:53:46.179822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:53:46.179822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:53:46.180964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ddb91-af9e-4b83-b5de-8ab74550f093 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:46.200491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 740/1478] 10.222.0.72 () {56 vars in 1108 bytes} [Mon Sep 26 17:53:45 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:53:48.409307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40faac9-b81d-47bb-a04f-61ac50545a22 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:48.409841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40faac9-b81d-47bb-a04f-61ac50545a22 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:48.409841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40faac9-b81d-47bb-a04f-61ac50545a22 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:48.411262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 739/1479] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:48.416634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166408a-6a02-49ea-b5b6-cc01a6ebf69f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:48.417042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166408a-6a02-49ea-b5b6-cc01a6ebf69f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:48.417193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166408a-6a02-49ea-b5b6-cc01a6ebf69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:48.488230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5166408a-6a02-49ea-b5b6-cc01a6ebf69f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:48.489462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5166408a-6a02-49ea-b5b6-cc01a6ebf69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:48.587121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 741/1480] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:48.898282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40faac9-b81d-47bb-a04f-61ac50545a22 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:48.899827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c40faac9-b81d-47bb-a04f-61ac50545a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:48.909599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:49.046683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40faac9-b81d-47bb-a04f-61ac50545a22 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:49.128650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:49.161683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592010f-491a-491b-939b-388546909d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:49.162213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592010f-491a-491b-939b-388546909d00 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:49.162494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592010f-491a-491b-939b-388546909d00 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:49.162765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592010f-491a-491b-939b-388546909d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:49.164458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1592010f-491a-491b-939b-388546909d00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:49.255809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592010f-491a-491b-939b-388546909d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:49.257425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592010f-491a-491b-939b-388546909d00 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:49.275276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 740/1481] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:53:51.672337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575948bc-17b8-4ec5-ac2d-671ab385f646 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:51.672939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575948bc-17b8-4ec5-ac2d-671ab385f646 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:51.672939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575948bc-17b8-4ec5-ac2d-671ab385f646 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:51.675194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 742/1482] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:51.680606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac9268-9d3b-4381-b47f-e4710e510e05 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:51.681037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac9268-9d3b-4381-b47f-e4710e510e05 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:51.681156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac9268-9d3b-4381-b47f-e4710e510e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:51.747114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-50ac9268-9d3b-4381-b47f-e4710e510e05 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:51.748497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-50ac9268-9d3b-4381-b47f-e4710e510e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:51.794851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:51.847259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 741/1483] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:52.138864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575948bc-17b8-4ec5-ac2d-671ab385f646 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:52.140428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-575948bc-17b8-4ec5-ac2d-671ab385f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:52.164929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:52.246993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575948bc-17b8-4ec5-ac2d-671ab385f646 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:52.321999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:52.322597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:52.322902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:52.323175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:52.324763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:52.399263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:52.435345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:52.436900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba562227-9dcc-433f-89b7-ead50a29b654 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:52.455277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 743/1484] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:54.658949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497d104-cc2e-4d67-8122-357ca111f1ec None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:54.659674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497d104-cc2e-4d67-8122-357ca111f1ec None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:54.659674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497d104-cc2e-4d67-8122-357ca111f1ec None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:54.660840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 742/1485] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:53:54.666114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282758b-e173-40a5-8091-f7abbea4b6f6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:54.666523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282758b-e173-40a5-8091-f7abbea4b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:54.666658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282758b-e173-40a5-8091-f7abbea4b6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:54.733880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2282758b-e173-40a5-8091-f7abbea4b6f6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:54.735143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2282758b-e173-40a5-8091-f7abbea4b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:54.834751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 744/1486] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:55.128956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497d104-cc2e-4d67-8122-357ca111f1ec None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:55.130474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f497d104-cc2e-4d67-8122-357ca111f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:55.232851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497d104-cc2e-4d67-8122-357ca111f1ec None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:55.313303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:55.315694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:55.316322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:55.316805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:55.317249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:55.319169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:55.431340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:55.432803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf77-a7c1-437d-9ae5-bef77e568989 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:55.451063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 743/1487] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:53:57.787968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9ebf9-48a7-44f0-9a7f-5103f5cf525d None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:57.788515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9ebf9-48a7-44f0-9a7f-5103f5cf525d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:57.788515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9ebf9-48a7-44f0-9a7f-5103f5cf525d None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:57.789864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 745/1488] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:53:57.795114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9cacb-c2f8-45a7-a440-c2e18cdfee3f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:57.795665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9cacb-c2f8-45a7-a440-c2e18cdfee3f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:57.795665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9cacb-c2f8-45a7-a440-c2e18cdfee3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:57.863144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-dea9cacb-c2f8-45a7-a440-c2e18cdfee3f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:53:57.864351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-dea9cacb-c2f8-45a7-a440-c2e18cdfee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:57.915955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:57.959825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 744/1489] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:53:58.263817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9ebf9-48a7-44f0-9a7f-5103f5cf525d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:53:58.265414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2bc9ebf9-48a7-44f0-9a7f-5103f5cf525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:58.301176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:58.368808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9ebf9-48a7-44f0-9a7f-5103f5cf525d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:53:58.478087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:53:58.478690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:53:58.479000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:53:58.479273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:53:58.480856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:53:58.559764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:53:58.589849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:53:58.591315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d552f5-4b8b-4ae1-a767-06417ed2178b service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:53:58.610113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 746/1490] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:01.000990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf1a08-feaa-4a44-af3b-aa075a5b29f3 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:01.001489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf1a08-feaa-4a44-af3b-aa075a5b29f3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:01.001586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf1a08-feaa-4a44-af3b-aa075a5b29f3 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:01.003133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 745/1491] 10.222.0.72 () {54 vars in 927 bytes} [Mon Sep 26 17:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:01.008385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880067-8dd6-4380-bcb9-321b4557663f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:01.008797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880067-8dd6-4380-bcb9-321b4557663f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:01.008934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880067-8dd6-4380-bcb9-321b4557663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:01.077288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-85880067-8dd6-4380-bcb9-321b4557663f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:01.078513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-85880067-8dd6-4380-bcb9-321b4557663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:01.182729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 747/1492] 10.222.0.72 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:54:01.480089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf1a08-feaa-4a44-af3b-aa075a5b29f3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:01.481686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b5bf1a08-feaa-4a44-af3b-aa075a5b29f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:01.586576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf1a08-feaa-4a44-af3b-aa075a5b29f3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:01.662725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:01.663353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:01.663649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:01.663907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:01.665569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:01.674998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:01.779712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:01.781139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1c57b-e065-4cf9-915c-09765aade543 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:01.799595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 746/1493] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:11.181189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=82 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:11.797099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=50 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:15.332890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:15.333460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:15.333460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:15.411725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:15.413006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:15.520414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 748/1494] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:15.528663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:15.529152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:15.529256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:15.606860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:15.607998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:15.707785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 747/1495] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:15.712606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:15.713841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:15.726672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13d5c0-6931-46ab-951d-2b7c328a54b2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:15.788773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:15.822882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:15.823517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:15.823818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:15.824092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:15.824741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:55:15.825068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:15.826082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:15.826082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:55:15.826082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:55:15.826996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:15.839012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 749/1496] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:55:15.848417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:15.857730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e60310-3951-447a-b5f1-c3a86908a71b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:15.921288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:15.956702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:15.957314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:15.957590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:15.957861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:15.958500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:15.960262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:15.960262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:55:15.960262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:55:15.961445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:16.020207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 748/1497] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:55:15 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:55:16.024685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:16.026304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.060296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d037b2-b490-4a69-993b-69f97324d765 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:16.158228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:16.158886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.159175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.159443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.160071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:16.161112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:16.161112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:55:16.161112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:55:16.162307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:16.205997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 750/1498] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:55:16 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:55:16.210186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:16.211714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.275310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfbb50-da07-4cea-88a9-ee9e62ec114c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:16.357002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:16.367889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:16.368486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.368759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.369025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.369656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:16.370657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:16.370657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:55:16.370657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:55:16.371565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:16.384343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 749/1499] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:55:16.388539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:16.390083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.402247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added885-e253-41ad-a960-4c53e37f238c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:16.427776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:16.498462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:16.499093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.499371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.499638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.500268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:16.502111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:16.502111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:55:16.502111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:55:16.503023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cb5c1-f557-4dec-a4da-769be4914559 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:16.515188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 751/1500] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:55:16.519477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:16.520995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.554342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca8b8a-f4be-40a4-b073-edb2bbb0890d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:16.648412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:16.649074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.649356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.649632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] PATH_INFO: `/v3/projects/aa1e3ea2094e4fe989c8576bbb5070c8/users/ab02ef3d2794430ebcd737ed2fbd5989/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.671810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1e3ea2094e4fe989c8576bbb5070c8, user_id=ab02ef3d2794430ebcd737ed2fbd5989, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:16.673512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:16.673512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:55:16.673512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:55:16.675493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91803a6-c49e-48e4-8e68-eb8df8e64339 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:16.700418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 750/1501] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:55:16 2022] PUT /identity/v3/projects/aa1e3ea2094e4fe989c8576bbb5070c8/users/ab02ef3d2794430ebcd737ed2fbd5989/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:55:16.712581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aff909-b6e9-40ae-aa09-79fceafad35a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.712868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aff909-b6e9-40ae-aa09-79fceafad35a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.713147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aff909-b6e9-40ae-aa09-79fceafad35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.771049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-82aff909-b6e9-40ae-aa09-79fceafad35a None None] MFA Rules not processed for user `ab02ef3d2794430ebcd737ed2fbd5989`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:16.772247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-82aff909-b6e9-40ae-aa09-79fceafad35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.849582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 752/1502] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:16.863402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcbacb-2e19-4ce5-9b7e-3fb7218d5ef1 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.863694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcbacb-2e19-4ce5-9b7e-3fb7218d5ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.863974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcbacb-2e19-4ce5-9b7e-3fb7218d5ef1 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.865366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 751/1503] 10.222.0.72 () {54 vars in 963 bytes} [Mon Sep 26 17:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:16.870201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f444684-bc59-4f01-bea0-29af8fb4fe08 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.870489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f444684-bc59-4f01-bea0-29af8fb4fe08 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.870759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f444684-bc59-4f01-bea0-29af8fb4fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.902243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:16.947379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6f444684-bc59-4f01-bea0-29af8fb4fe08 None None] MFA Rules not processed for user `39bde61b0b7c4cf3950ef63248a3b0da`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:16.948599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6f444684-bc59-4f01-bea0-29af8fb4fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:17.024273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 753/1504] 10.222.0.72 () {58 vars in 1060 bytes} [Mon Sep 26 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:17.032529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcbacb-2e19-4ce5-9b7e-3fb7218d5ef1 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:17.034063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-50dcbacb-2e19-4ce5-9b7e-3fb7218d5ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:17.083634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:17.140729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcbacb-2e19-4ce5-9b7e-3fb7218d5ef1 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:17.214468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:17.215091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:17.215369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:17.215634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:17.217214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:17.315578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:17.316916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:17.320349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:17.336091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 752/1505] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:17.383673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eeff2-d834-402d-9a49-c73585d833a0 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:17.383957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eeff2-d834-402d-9a49-c73585d833a0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:17.384228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eeff2-d834-402d-9a49-c73585d833a0 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:17.385514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 754/1506] 10.222.0.72 () {52 vars in 830 bytes} [Mon Sep 26 17:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:17.389395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:17.390686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:17.489997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f18d-8124-45e5-8ecd-9e7936584e30 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:17.585965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:17.586617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:17.586780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:17.587083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] PATH_INFO: `/v3/users/ab02ef3d2794430ebcd737ed2fbd5989` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:17.600115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab02ef3d2794430ebcd737ed2fbd5989)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:17.601132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:17.601132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:55:17.601132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:55:17.602493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:17.612679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:17.672509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab02ef3d2794430ebcd737ed2fbd5989'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:17.673224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab02ef3d2794430ebcd737ed2fbd5989'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:17.686688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab02ef3d2794430ebcd737ed2fbd5989'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:17.710801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab02ef3d2794430ebcd737ed2fbd5989'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:17.711298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab02ef3d2794430ebcd737ed2fbd5989'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:17.726410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 753/1507] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:55:17 2022] DELETE /identity/v3/users/ab02ef3d2794430ebcd737ed2fbd5989 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:55:17.730987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:17.731498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:17.731606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:17.789741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:17.791006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:17.869212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:17.882539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 755/1508] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:17.893394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:17.894635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:17.996683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec276-6256-43e9-8f56-4da69a6ab897 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:18.052459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:18.073181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:18.073788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:18.074063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:18.074327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:18.075943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:18.184072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:18.185423 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9749d-4aac-4480-a406-233fe347c5c8 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:18.204506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 754/1509] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:18.539946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:18.541170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:18.553242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc42f23-1490-40e8-b294-e9400bd82916 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:18.667832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:18.668464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:18.668750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:18.669022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] PATH_INFO: `/v3/projects/aa1e3ea2094e4fe989c8576bbb5070c8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:18.679214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1e3ea2094e4fe989c8576bbb5070c8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:18.680244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:55:18.680244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:55:18.680244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:55:18.681459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:18.740862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1e3ea2094e4fe989c8576bbb5070c8', 'request_id': 'req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:18.754544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1e3ea2094e4fe989c8576bbb5070c8', 'request_id': 'req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:18.764091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1e3ea2094e4fe989c8576bbb5070c8', 'request_id': 'req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:18.788261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] The token cache is being invalidate because project aa1e3ea2094e4fe989c8576bbb5070c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:55:18.788538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:18.788968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d312d085-a59f-4ae8-9b93-a98c9dc1c1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:55:18.790372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 756/1510] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:55:18 2022] DELETE /identity/v3/projects/aa1e3ea2094e4fe989c8576bbb5070c8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:55:28.201581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=62 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:28.786131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:53.117481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e863e0-e1f6-4734-834a-f41de9cdbbb5 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.118069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e863e0-e1f6-4734-834a-f41de9cdbbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.118069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e863e0-e1f6-4734-834a-f41de9cdbbb5 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.119436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:53.124433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7481cd-2a5a-4f76-bf31-b433f08421cf None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.125501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7481cd-2a5a-4f76-bf31-b433f08421cf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.125501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7481cd-2a5a-4f76-bf31-b433f08421cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.208917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-eb7481cd-2a5a-4f76-bf31-b433f08421cf None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:53.210158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-eb7481cd-2a5a-4f76-bf31-b433f08421cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:53.315720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:53.483538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e863e0-e1f6-4734-834a-f41de9cdbbb5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:53.486261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-83e863e0-e1f6-4734-834a-f41de9cdbbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:53.610085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e863e0-e1f6-4734-834a-f41de9cdbbb5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:53.615524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee540-5136-4296-9123-f211fda5f9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:53.616133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee540-5136-4296-9123-f211fda5f9d6 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.616413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee540-5136-4296-9123-f211fda5f9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.616688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee540-5136-4296-9123-f211fda5f9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.625421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caee540-5136-4296-9123-f211fda5f9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:53.626991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caee540-5136-4296-9123-f211fda5f9d6 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:53.636329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 756/1513] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:53.904078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f9eab-4ac3-44ee-b7ed-c9b44ddc77c3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.904357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f9eab-4ac3-44ee-b7ed-c9b44ddc77c3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.904669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f9eab-4ac3-44ee-b7ed-c9b44ddc77c3 None None] PATH_INFO: `/v3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.906256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 758/1514] 10.222.0.98 () {54 vars in 925 bytes} [Mon Sep 26 17:56:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:56:53.913461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2fb51-70e7-458d-96b6-31b79bc14eb7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.913758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2fb51-70e7-458d-96b6-31b79bc14eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.914054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2fb51-70e7-458d-96b6-31b79bc14eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.978788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c2a2fb51-70e7-458d-96b6-31b79bc14eb7 None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:53.980492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c2a2fb51-70e7-458d-96b6-31b79bc14eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:54.048491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:54.054069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 757/1515] 10.222.0.98 () {58 vars in 1015 bytes} [Mon Sep 26 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:54.771777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eeaca-4b00-456c-a6b3-a276d0a349d5 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:54.772341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eeaca-4b00-456c-a6b3-a276d0a349d5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:54.772341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710eeaca-4b00-456c-a6b3-a276d0a349d5 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:54.773671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 759/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:54.779631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ed285-4178-432d-b802-3f84ee7a2ed8 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:54.780038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ed285-4178-432d-b802-3f84ee7a2ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:54.780176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ed285-4178-432d-b802-3f84ee7a2ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:54.800051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fc1ed285-4178-432d-b802-3f84ee7a2ed8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:54.801262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fc1ed285-4178-432d-b802-3f84ee7a2ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:54.809288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 758/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:54.961740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eeaca-4b00-456c-a6b3-a276d0a349d5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:54.976462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710eeaca-4b00-456c-a6b3-a276d0a349d5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:54.981999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ca06d-3370-41e9-9022-bf0ea109b5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:54.982617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ca06d-3370-41e9-9022-bf0ea109b5a0 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:54.982913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ca06d-3370-41e9-9022-bf0ea109b5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:54.983191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ca06d-3370-41e9-9022-bf0ea109b5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:54.992188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ca06d-3370-41e9-9022-bf0ea109b5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:54.993778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ca06d-3370-41e9-9022-bf0ea109b5a0 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:55.004053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 760/1518] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:04.794008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:04.998997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=75 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:05.686283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503498f5-842d-48a5-bff8-da826f42403f None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:05.686575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503498f5-842d-48a5-bff8-da826f42403f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:05.686948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503498f5-842d-48a5-bff8-da826f42403f None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:05.688837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 759/1519] 10.222.0.72 () {54 vars in 961 bytes} [Mon Sep 26 17:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:57:05.694290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b8c8d-aae2-47ce-92fd-140983ab8d6c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:05.694579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b8c8d-aae2-47ce-92fd-140983ab8d6c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:05.694894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b8c8d-aae2-47ce-92fd-140983ab8d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:05.715392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1f9b8c8d-aae2-47ce-92fd-140983ab8d6c None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:05.716599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1f9b8c8d-aae2-47ce-92fd-140983ab8d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:05.726114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 761/1520] 10.222.0.72 () {58 vars in 1058 bytes} [Mon Sep 26 17:57:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:05.733909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503498f5-842d-48a5-bff8-da826f42403f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:05.747278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503498f5-842d-48a5-bff8-da826f42403f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:05.752290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b61ac-935f-493b-a5ed-5dadedf3b229 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:05.752885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b61ac-935f-493b-a5ed-5dadedf3b229 service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:05.753244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b61ac-935f-493b-a5ed-5dadedf3b229 service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:05.753532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b61ac-935f-493b-a5ed-5dadedf3b229 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:05.762173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b61ac-935f-493b-a5ed-5dadedf3b229 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:05.763770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b61ac-935f-493b-a5ed-5dadedf3b229 service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:05.774001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 760/1521] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:06.495761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad8e38-5ec7-49c2-bff6-a57196dcd739 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:06.496327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad8e38-5ec7-49c2-bff6-a57196dcd739 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:06.496327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad8e38-5ec7-49c2-bff6-a57196dcd739 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:06.497687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:06.504624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f339b-f25c-4622-9154-d34954be6753 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:06.505056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f339b-f25c-4622-9154-d34954be6753 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:06.505199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f339b-f25c-4622-9154-d34954be6753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:06.524425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-8a9f339b-f25c-4622-9154-d34954be6753 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:06.525642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8a9f339b-f25c-4622-9154-d34954be6753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:06.533775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:06.687592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad8e38-5ec7-49c2-bff6-a57196dcd739 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:06.702342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad8e38-5ec7-49c2-bff6-a57196dcd739 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:06.708240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed79a3-1332-46a3-9d8e-ac84a4e48424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:06.708838 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed79a3-1332-46a3-9d8e-ac84a4e48424 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:06.709117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed79a3-1332-46a3-9d8e-ac84a4e48424 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:06.709388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed79a3-1332-46a3-9d8e-ac84a4e48424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:06.717636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed79a3-1332-46a3-9d8e-ac84a4e48424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:06.719389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed79a3-1332-46a3-9d8e-ac84a4e48424 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:06.728819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 763/1524] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:07.830143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a06c7-40af-4adb-bce1-efef94112cd8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:07.830745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a06c7-40af-4adb-bce1-efef94112cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:07.830745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a06c7-40af-4adb-bce1-efef94112cd8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:07.832093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:07.838717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247afc2-e16e-4f95-a2d4-f5ba9665b3af None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:07.839216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247afc2-e16e-4f95-a2d4-f5ba9665b3af None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:07.839325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247afc2-e16e-4f95-a2d4-f5ba9665b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:07.859587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9247afc2-e16e-4f95-a2d4-f5ba9665b3af None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:07.860793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9247afc2-e16e-4f95-a2d4-f5ba9665b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:07.868561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:08.025955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a06c7-40af-4adb-bce1-efef94112cd8 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:08.041382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a06c7-40af-4adb-bce1-efef94112cd8 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:08.047002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868d4db-8338-4407-a549-b3146160ad37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:08.047594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868d4db-8338-4407-a549-b3146160ad37 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:08.047879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868d4db-8338-4407-a549-b3146160ad37 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:08.048151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868d4db-8338-4407-a549-b3146160ad37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:08.056165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868d4db-8338-4407-a549-b3146160ad37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:08.057767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868d4db-8338-4407-a549-b3146160ad37 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:08.067058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 763/1527] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:12.269307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c44e-d303-430c-92d5-55af9ab00b49 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:12.269880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c44e-d303-430c-92d5-55af9ab00b49 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:12.269880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c44e-d303-430c-92d5-55af9ab00b49 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:12.271223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:57:12.276315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b7d5-01c3-4ed2-968d-c91602edf786 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:12.276731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b7d5-01c3-4ed2-968d-c91602edf786 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:12.276854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b7d5-01c3-4ed2-968d-c91602edf786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:12.298068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-75e3b7d5-01c3-4ed2-968d-c91602edf786 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:12.299437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-75e3b7d5-01c3-4ed2-968d-c91602edf786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:12.307738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:12.612328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c44e-d303-430c-92d5-55af9ab00b49 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:12.614097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aa79c44e-d303-430c-92d5-55af9ab00b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:12.695421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c44e-d303-430c-92d5-55af9ab00b49 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:12.699301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ed1cf-d1fb-44f0-8f12-e7944fcda369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:12.699904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed1cf-d1fb-44f0-8f12-e7944fcda369 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:12.700190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed1cf-d1fb-44f0-8f12-e7944fcda369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:12.700468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed1cf-d1fb-44f0-8f12-e7944fcda369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:12.709719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ed1cf-d1fb-44f0-8f12-e7944fcda369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:12.711245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ed1cf-d1fb-44f0-8f12-e7944fcda369 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:12.720625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 766/1530] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:14.855435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5291e-e3fd-4812-b1bd-66cd7ff70962 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:14.856004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5291e-e3fd-4812-b1bd-66cd7ff70962 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:14.856004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5291e-e3fd-4812-b1bd-66cd7ff70962 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:14.857387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:14.863504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661fee-3b42-4af5-82d1-890da817e637 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:14.863911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661fee-3b42-4af5-82d1-890da817e637 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:14.864052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661fee-3b42-4af5-82d1-890da817e637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:14.884225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-18661fee-3b42-4af5-82d1-890da817e637 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:14.885422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-18661fee-3b42-4af5-82d1-890da817e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:14.893257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:15.192350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5291e-e3fd-4812-b1bd-66cd7ff70962 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:15.193767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-19b5291e-e3fd-4812-b1bd-66cd7ff70962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:15.207828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5291e-e3fd-4812-b1bd-66cd7ff70962 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:15.211395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a07be-159a-4055-aeb1-89c868cdec24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:15.212004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a07be-159a-4055-aeb1-89c868cdec24 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:15.212264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a07be-159a-4055-aeb1-89c868cdec24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:15.212535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a07be-159a-4055-aeb1-89c868cdec24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:15.221627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a07be-159a-4055-aeb1-89c868cdec24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:15.223070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a07be-159a-4055-aeb1-89c868cdec24 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:15.232258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 766/1533] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:19.384633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6bce7-6627-47d4-b0e1-3515f54360e3 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:19.385220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6bce7-6627-47d4-b0e1-3515f54360e3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:19.385220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6bce7-6627-47d4-b0e1-3515f54360e3 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:19.386487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:19.391980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c61cf4-08d4-4e82-9737-66ef011de790 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:19.392421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c61cf4-08d4-4e82-9737-66ef011de790 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:19.392527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c61cf4-08d4-4e82-9737-66ef011de790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:19.411932 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c2c61cf4-08d4-4e82-9737-66ef011de790 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:19.413140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c2c61cf4-08d4-4e82-9737-66ef011de790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:19.420928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:19.733360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6bce7-6627-47d4-b0e1-3515f54360e3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:19.736586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e6a6bce7-6627-47d4-b0e1-3515f54360e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:19.813899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6bce7-6627-47d4-b0e1-3515f54360e3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:19.817446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656de802-cced-4453-935d-4c7b74c30ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:19.818046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656de802-cced-4453-935d-4c7b74c30ebd service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:19.818323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656de802-cced-4453-935d-4c7b74c30ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:19.818592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656de802-cced-4453-935d-4c7b74c30ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:19.827693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656de802-cced-4453-935d-4c7b74c30ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:19.829119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656de802-cced-4453-935d-4c7b74c30ebd service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:19.838468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 769/1536] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:22.011921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe60579-3599-46c6-a325-a60cf7105d27 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:22.012416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe60579-3599-46c6-a325-a60cf7105d27 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:22.012544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe60579-3599-46c6-a325-a60cf7105d27 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:22.013805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:22.019633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0259c7-debc-40a0-82de-e2c3cea42376 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:22.020080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0259c7-debc-40a0-82de-e2c3cea42376 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:22.020189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0259c7-debc-40a0-82de-e2c3cea42376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:22.039834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7b0259c7-debc-40a0-82de-e2c3cea42376 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:22.041041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7b0259c7-debc-40a0-82de-e2c3cea42376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:22.049064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:22.355127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe60579-3599-46c6-a325-a60cf7105d27 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:22.356793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2fe60579-3599-46c6-a325-a60cf7105d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:22.369681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe60579-3599-46c6-a325-a60cf7105d27 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:22.373158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90564a11-5007-4a7e-aa11-b9d0a14d31ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:22.373746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564a11-5007-4a7e-aa11-b9d0a14d31ff service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:22.374023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564a11-5007-4a7e-aa11-b9d0a14d31ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:22.374429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90564a11-5007-4a7e-aa11-b9d0a14d31ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:22.383216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90564a11-5007-4a7e-aa11-b9d0a14d31ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:22.384616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90564a11-5007-4a7e-aa11-b9d0a14d31ff service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:22.394257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 769/1539] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:32.035076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:32.389702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=33 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:49.245460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.245460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.246879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.287384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.288623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.303019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 771/1540] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:58:49.313744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.314357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.314357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.340070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.341367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.350295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 770/1541] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:49.355666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.373269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7497b8-6af9-4056-bf42-89f364ca1da2 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.380721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.381373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.381655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.381931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.382618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:49.382959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.384637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.402357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 772/1542] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:49.410768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.420595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe15563-cfff-4ce8-b1a4-51aba2d26dcf None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.431614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d24072-5356-4b9e-b077-b7990a6cf8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.432245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d24072-5356-4b9e-b077-b7990a6cf8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.432521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d24072-5356-4b9e-b077-b7990a6cf8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.432794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d24072-5356-4b9e-b077-b7990a6cf8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.433434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d24072-5356-4b9e-b077-b7990a6cf8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.435382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d24072-5356-4b9e-b077-b7990a6cf8ac admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.501689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 771/1543] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/projects => generated 391 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:58:49.505956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.525540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d51df-88cd-4403-9ca8-11ffbca56d2f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.614947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a020ae-cc10-4227-bd99-75668c83cace None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.615752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a020ae-cc10-4227-bd99-75668c83cace None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.615752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a020ae-cc10-4227-bd99-75668c83cace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.622625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.623355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.623656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.623926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.624540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.627262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.648471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-82a020ae-cc10-4227-bd99-75668c83cace None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.649873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-82a020ae-cc10-4227-bd99-75668c83cace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.667272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 773/1545] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:49.667509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 772/1545] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:49.672069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.679243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.679559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.679788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.682645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e58e1-5d58-4ed9-ab15-ac1befc75218 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.698010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859be268-257a-48e7-8af2-9adaf46b5b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.698812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859be268-257a-48e7-8af2-9adaf46b5b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.699126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859be268-257a-48e7-8af2-9adaf46b5b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.699404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859be268-257a-48e7-8af2-9adaf46b5b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.700061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859be268-257a-48e7-8af2-9adaf46b5b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.702057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859be268-257a-48e7-8af2-9adaf46b5b9a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.709659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.710888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.721119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 774/1546] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:49.722478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.723616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.723616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.733981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 773/1547] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:58:49.735291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.748346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213c23-0c03-4f3b-a099-aa98b4f8e29d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.761541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.762708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.765432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.766684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.766987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.767267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.768002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.769597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.795285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 774/1548] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:49.796221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.801061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 775/1549] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:49.801902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:49.812204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a103da-bfb5-4f50-a1dd-85f6a1751f39 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.820868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.821485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.821761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.822029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.822958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:49.823262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.824265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:58:49.824265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:58:49.824265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:58:49.825109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4685-3f35-41ae-b356-e64e0f3effc8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:49.825237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3a473-d6a4-41fe-87dc-d78050407c54 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.838426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 775/1550] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:49.839548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328aeb2-e117-476e-98e9-3ed5a518a411 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.839829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328aeb2-e117-476e-98e9-3ed5a518a411 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.840096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328aeb2-e117-476e-98e9-3ed5a518a411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.840600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:49.842650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.842650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.842650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] PATH_INFO: `/v3/projects/93579356e4b54de5a06d395cd499ca13/users/008e6c2c897741889510e6bf5181a9c6/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.860017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93579356e4b54de5a06d395cd499ca13, user_id=008e6c2c897741889510e6bf5181a9c6, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:49.860252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c328aeb2-e117-476e-98e9-3ed5a518a411 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.861076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:58:49.861076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:58:49.861076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:58:49.861437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c328aeb2-e117-476e-98e9-3ed5a518a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.863036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f184c-31a6-44a3-9f10-6810c2ea6cc8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:49.869996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 776/1551] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:49.871367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca6911-60ae-4164-83ed-0130018b0d40 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.871648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca6911-60ae-4164-83ed-0130018b0d40 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.871949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca6911-60ae-4164-83ed-0130018b0d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.884294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 776/1552] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:49 2022] PUT /identity/v3/projects/93579356e4b54de5a06d395cd499ca13/users/008e6c2c897741889510e6bf5181a9c6/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:49.888001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c83e6c-3f3b-4217-8d32-1cf82c91cfc5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.888001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c83e6c-3f3b-4217-8d32-1cf82c91cfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.888001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c83e6c-3f3b-4217-8d32-1cf82c91cfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.895352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f9ca6911-60ae-4164-83ed-0130018b0d40 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.896753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f9ca6911-60ae-4164-83ed-0130018b0d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.906941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-07c83e6c-3f3b-4217-8d32-1cf82c91cfc5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:49.908594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-07c83e6c-3f3b-4217-8d32-1cf82c91cfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:49.976090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 777/1553] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:49.977688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.978300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.978580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:49.986515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 777/1554] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:49.987913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27648-dcbc-42b8-b9d2-3a410621f222 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:49.988200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27648-dcbc-42b8-b9d2-3a410621f222 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:49.988474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27648-dcbc-42b8-b9d2-3a410621f222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.001831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.003083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.008555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-acd27648-dcbc-42b8-b9d2-3a410621f222 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.009726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-acd27648-dcbc-42b8-b9d2-3a410621f222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.011241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 778/1555] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.012191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.021848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef93a6-7380-4f31-95de-039798770e43 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.024965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 778/1556] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.026202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec71833-5088-404e-9999-000fbcbc242f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.026408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9d961-3992-4215-9753-e62a21f1c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.026546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec71833-5088-404e-9999-000fbcbc242f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.026746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec71833-5088-404e-9999-000fbcbc242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.027030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9d961-3992-4215-9753-e62a21f1c5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.027315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9d961-3992-4215-9753-e62a21f1c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.027588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9d961-3992-4215-9753-e62a21f1c5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.028216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9d961-3992-4215-9753-e62a21f1c5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.030102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9d961-3992-4215-9753-e62a21f1c5a6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.047958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1ec71833-5088-404e-9999-000fbcbc242f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.049350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1ec71833-5088-404e-9999-000fbcbc242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.057539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 779/1557] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.065301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87ee5-9d9c-4b3f-9bde-d803a5520c3f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.065435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 779/1558] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.065532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87ee5-9d9c-4b3f-9bde-d803a5520c3f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.065532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87ee5-9d9c-4b3f-9bde-d803a5520c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.065685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.065685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.065685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.080164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9cb87ee5-9d9c-4b3f-9bde-d803a5520c3f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.080828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.081371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9cb87ee5-9d9c-4b3f-9bde-d803a5520c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.082008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.090646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:50.129673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:50.144787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 780/1559] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.148120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 780/1560] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.148120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.148346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06b55d-62ba-42de-b0da-3e90fd972dba None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.148346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06b55d-62ba-42de-b0da-3e90fd972dba None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.148346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06b55d-62ba-42de-b0da-3e90fd972dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.163174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f56fe7-2447-4f19-8da6-e43e5386c906 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.166631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c06b55d-62ba-42de-b0da-3e90fd972dba None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.167851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.168028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c06b55d-62ba-42de-b0da-3e90fd972dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.168459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.168742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.169019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.169661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:50.169969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.171601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f63a2-d6b2-4cc3-9e6e-b22c002e3dec admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.177003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 781/1561] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.180903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.180903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.180903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.194150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 781/1562] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:50.194150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.194150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.194150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.199335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.200537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.208419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 782/1563] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.209199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.209900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.211134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.218666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182f3d9-aa7c-4e30-aa40-7e5cd28fe510 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.220155 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 782/1564] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.221341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.227374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.228066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.228355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.228632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.229279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:50.229587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.230974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6f7da-8f46-4010-af13-a1700ba29eeb None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.231389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d90f0-210e-4966-9bf1-31af052fc787 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.236851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.237504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.237789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.238064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.238691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.239735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:58:50.239735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:58:50.239735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:58:50.240998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.244131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 783/1565] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:50.244131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.244131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.244131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.265667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 783/1566] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.266438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.270233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:50.271470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:50.279510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 784/1567] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:50.284144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.295508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053cba3-00bd-4ffd-b0f5-4c351833e3a0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.303176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.304027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.306495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.306495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.306495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:50.306495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.307246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.322028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 785/1568] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:50.323317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.336828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ed80e-2268-4ea8-af57-4bc2499ec9ca admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.337417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b579f-e91f-44ff-959d-20f89f7c036c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.340636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.341250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.341547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.341936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.343378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.344925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.345216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.345491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.346170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.347209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:58:50.347209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:58:50.347209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:58:50.348772 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.351826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.353232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.365479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 784/1569] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:50.366394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.377428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d468ce-6f99-4b44-95e3-4888aa62ab02 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.382329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.383064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.383298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.383518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.384326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:50.384534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.385180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 786/1570] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.386122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.386330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.400328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 785/1571] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:50.401180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.411816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f1530-5a63-4a3a-bad5-6eb2d9f5ee00 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.477792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80113a-7c7c-46e5-851f-6a09ec17288a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.482090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.482707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.482980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.483115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.483311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.483602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.483939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.484069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.484267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.485801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:50.486196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.487034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.487958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.507734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 787/1572] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:50.507734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.512606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e637e-49a7-4fb0-b763-e043c3b7b833 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.518928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.519201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.519656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.519996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.521243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.522892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.527131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 786/1573] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.527131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.539633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 788/1574] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:50.539633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.554932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dd84e-9a79-4120-a021-828ba43f776b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.564998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:50.596884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:50.609794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe8968-86f1-4307-87b0-ebfba378fd8a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.615863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.616698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.616987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.617305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.617976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:58:50.618336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.620521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.631791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.632413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.632699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.632974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.633616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.635613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.636102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 787/1575] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:58:50.636936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.647838 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8aa858-1121-4b85-8e70-dc2c148a1aeb admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.653378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.654148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.654497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.654848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.655626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.658182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.682899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 789/1576] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.682899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.691413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 788/1577] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/users => generated 289 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.691413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.706515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22e21-37bc-4a45-af59-49689ba65d12 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.758750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcdc32-9e60-4fef-bb1d-e7b55101fc7a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.764343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.765082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.766502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.766502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.767412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.769471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.795209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.798232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.799203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.800100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.803611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.805580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.808490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 790/1578] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.809454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.840618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 789/1579] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.840618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.901123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e008dd1-5ea0-4560-95e4-6884d65359fd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.907536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.908407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.908684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.908957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.910444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.915991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.920521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d8b85-d17d-4190-9375-fae7d820a41a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.928111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.929737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.930438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.930795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.931508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.933358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.941671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 791/1580] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.942791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.948605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 790/1581] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:50.949341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:50.957415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6f2e-432b-4d0d-bbd8-1ca6efe5b013 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.960007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319ba74-e849-4c3d-b092-5879092dd5f6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:50.963232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.963877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.964159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.964576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.964808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:50.965214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.965516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:50.965799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:50.966080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:50.966897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:50.968642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.969542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:50.998219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 791/1582] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:50.999779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.015052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f7dff-60d1-4d19-a22a-b69286588b7e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.015282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 792/1583] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:50 2022] POST /identity/v3/projects => generated 405 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:51.015282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.029694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:51.032235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:51.103260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0aff85-4828-4997-b084-3ba149defe22 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.103603 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.104296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.104579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.104859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.105599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.107347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.109254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.110670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.111123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.111409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.112057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.115014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.118910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 792/1584] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.119577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.133608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629750-b5f2-4b8a-8a60-29644a8eb297 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.139926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.141540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.141658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.141961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.142682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.145499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.148389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 793/1585] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:51 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:51.158877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.174409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 793/1586] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:51 2022] POST /identity/v3/users => generated 291 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:51.175309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.178140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87534a45-34de-48c6-9e61-d2e5d18fe6e6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.183292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.183779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.184060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.184338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.184990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.186765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.186926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376817f1-a41f-4f92-b2e1-868ce7bbf1ea admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.194508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.195482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.195790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.196094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] PATH_INFO: `/v3/projects/3f9772c3904046ee9778a870f4adc0ae/users/276174ef02b1409b840a0de13aa4095d/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.201151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 794/1587] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.205858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.217443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9772c3904046ee9778a870f4adc0ae, user_id=276174ef02b1409b840a0de13aa4095d, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.217813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297a054-d732-485e-803f-45db9f28cfe4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.220805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.224116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.224740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.225230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.225527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.226183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.228095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.240869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 795/1588] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.243809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.251808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 794/1589] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:51 2022] PUT /identity/v3/projects/3f9772c3904046ee9778a870f4adc0ae/users/276174ef02b1409b840a0de13aa4095d/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:51.253038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.263405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e382a-79ac-4bf3-bb2b-cb5e7e83f57a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.338047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.339650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.340116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.340721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.341664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.343338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1117fc7-2b54-49ad-b277-10ba8e21e9c6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.344749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.349017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.349768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.350054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.350331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.351499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.353246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.372665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 795/1590] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.373726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.383134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 796/1591] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:51 2022] POST /identity/v3/users => generated 303 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:51.385235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.390050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dc76-26dd-48d4-8541-922d697c44bc admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.397892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.399253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.399542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.399826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.400250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2b3b8-e814-402e-b162-26e4fbd315b6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.400481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.403737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.408895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.409614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.409969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.410319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.411444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.413595 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.418811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 796/1592] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.420136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.435585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5458fef-456f-49cc-a831-584ebab1aa2b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.438567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 797/1593] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.439825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.445478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbba88-3c3e-45c1-a958-86ceda1586a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.446470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbba88-3c3e-45c1-a958-86ceda1586a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.447037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbba88-3c3e-45c1-a958-86ceda1586a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.447830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbba88-3c3e-45c1-a958-86ceda1586a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.449016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbba88-3c3e-45c1-a958-86ceda1586a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.452139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbba88-3c3e-45c1-a958-86ceda1586a5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.454815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97908-8adf-45fb-a51f-80fc34574aa4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.466116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 797/1594] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.471183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.471668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.472161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.476198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.480216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.480604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.481605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.482735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.486711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.497570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:51.499945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:51.504308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 798/1595] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.505315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.515912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 798/1596] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:51.517130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.527774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125270e-bd7e-476e-b704-374b72909104 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.532889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c448b7a-341f-4393-ad10-ebc5a38ce2a6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.534787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.535703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.536033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.536401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] PATH_INFO: `/v3/projects/993e0f8cdcb744279b2987d0d8f321ae/users/28195a63990344edb6ea31f1f4463a82/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.540257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.540825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.541157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.542220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.543143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.545953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.558590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993e0f8cdcb744279b2987d0d8f321ae, user_id=28195a63990344edb6ea31f1f4463a82, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.559546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 799/1597] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.560744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.563583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.572978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31c0a-4214-4a2d-b878-9a84b8cf67b6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.579470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.580394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.580664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.580935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.583249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.583513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.587708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 799/1598] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:51 2022] PUT /identity/v3/projects/993e0f8cdcb744279b2987d0d8f321ae/users/28195a63990344edb6ea31f1f4463a82/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:51.588375 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.595664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 800/1599] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.597200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.612543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36620-a3c7-43ae-a08a-831bc77d15e0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.618207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:51.626571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:51.683918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa80471-9318-4aa4-8c1b-24b29e6c3d63 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.691153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.691849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.692183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.692553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.693410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.695247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.698614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.699585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.699973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.700366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] PATH_INFO: `/v3/projects/41f7e2a3f28f4734ae623883e0c53aeb/users/d667778c6e7b4879bc6c65dc82e230d1/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.714070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 800/1600] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.715268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.722317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f7e2a3f28f4734ae623883e0c53aeb, user_id=d667778c6e7b4879bc6c65dc82e230d1, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.726030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.727384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9738b9-3e95-4f7e-bb28-7d26b8a38dbd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.736191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.736789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.737208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.737488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] PATH_INFO: `/v3/projects/3f1770ed75fd48d098bb1efc839ebc0d/users/4bbd5a38a6ce472790128a37f62d472b/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.746375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 801/1601] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:51 2022] PUT /identity/v3/projects/41f7e2a3f28f4734ae623883e0c53aeb/users/d667778c6e7b4879bc6c65dc82e230d1/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:51.747652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.763496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1770ed75fd48d098bb1efc839ebc0d, user_id=4bbd5a38a6ce472790128a37f62d472b, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.766335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.808033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 801/1602] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:51 2022] PUT /identity/v3/projects/3f1770ed75fd48d098bb1efc839ebc0d/users/4bbd5a38a6ce472790128a37f62d472b/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:51.810593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.849069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886af53-9c74-4604-9f31-32fee3b2a38b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.915004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07621c9-26a8-4093-bf89-32f5924b6467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.915633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07621c9-26a8-4093-bf89-32f5924b6467 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.915918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07621c9-26a8-4093-bf89-32f5924b6467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.916197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07621c9-26a8-4093-bf89-32f5924b6467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.927445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07621c9-26a8-4093-bf89-32f5924b6467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.928817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ed73b-4647-403e-98bc-633695d52efd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.928962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07621c9-26a8-4093-bf89-32f5924b6467 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.934684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.935341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.935624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.936411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.937074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:51.940061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:51.940719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 802/1603] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:51.942857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.943142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.943421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.957148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 802/1604] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:51.957148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:51.968611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01130c0d-4cfb-499e-822e-6ca479f0d9e4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:51.974669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:51.975580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea233cd-d46c-4179-87c8-c4127fdda363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:51.976455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea233cd-d46c-4179-87c8-c4127fdda363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:51.976799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea233cd-d46c-4179-87c8-c4127fdda363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:51.977100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea233cd-d46c-4179-87c8-c4127fdda363 admin admin] PATH_INFO: `/v3/projects/aa42868f9e9e4d00be6a78ec54924712/users/8d562e923af0446db67404ce4ad36cc7/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:51.983644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:51.996808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 803/1605] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:51.997797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.002403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea233cd-d46c-4179-87c8-c4127fdda363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa42868f9e9e4d00be6a78ec54924712, user_id=8d562e923af0446db67404ce4ad36cc7, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:52.006310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea233cd-d46c-4179-87c8-c4127fdda363 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:52.008792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daeb3d-dd5c-47ef-9999-cf9da8241223 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:52.016433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de2ad-08f4-42e0-b419-58f88ae0cb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:52.017356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de2ad-08f4-42e0-b419-58f88ae0cb6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.017475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de2ad-08f4-42e0-b419-58f88ae0cb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.017775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de2ad-08f4-42e0-b419-58f88ae0cb6f admin admin] PATH_INFO: `/v3/projects/b7f0e15e9b6b48229b9f6d3cc0ee107b/users/6173cb93d6214493b75a91f2ae9e8a1f/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.029718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 803/1606] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:51 2022] PUT /identity/v3/projects/aa42868f9e9e4d00be6a78ec54924712/users/8d562e923af0446db67404ce4ad36cc7/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:52.029718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.029718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.029718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.041322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de2ad-08f4-42e0-b419-58f88ae0cb6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f0e15e9b6b48229b9f6d3cc0ee107b, user_id=6173cb93d6214493b75a91f2ae9e8a1f, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:52.046744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de2ad-08f4-42e0-b419-58f88ae0cb6f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:52.067225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] MFA Rules not processed for user `d667778c6e7b4879bc6c65dc82e230d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:52.068495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:52.082045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 804/1607] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:51 2022] PUT /identity/v3/projects/b7f0e15e9b6b48229b9f6d3cc0ee107b/users/6173cb93d6214493b75a91f2ae9e8a1f/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:52.086190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de4756-657f-433b-9b40-4f0354560650 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.086483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de4756-657f-433b-9b40-4f0354560650 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.088419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de4756-657f-433b-9b40-4f0354560650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.140197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-78de4756-657f-433b-9b40-4f0354560650 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:52.142096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-78de4756-657f-433b-9b40-4f0354560650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:52.167701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:52.180369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:52.199839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 804/1608] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:52.201290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.266160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 805/1609] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:52.271060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de4756-657f-433b-9b40-4f0354560650 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.339512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de4756-657f-433b-9b40-4f0354560650 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:52.344543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c87f1-cddd-4cc0-9f8c-5a69ca402802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:52.345389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c87f1-cddd-4cc0-9f8c-5a69ca402802 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.345495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c87f1-cddd-4cc0-9f8c-5a69ca402802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.345720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c87f1-cddd-4cc0-9f8c-5a69ca402802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.346316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123fbe0-799d-463c-abcd-5465adce35c8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:52.352231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f7b8-f460-47ee-9835-452aefde4593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:52.353139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f7b8-f460-47ee-9835-452aefde4593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.353588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f7b8-f460-47ee-9835-452aefde4593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.354053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f7b8-f460-47ee-9835-452aefde4593 admin admin] PATH_INFO: `/v3/projects/373eb8dd7c4447bf983907b3e73014a1/users/9f4bff49703c4c549ece69fc45e3cdc9/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.356842 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c87f1-cddd-4cc0-9f8c-5a69ca402802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:52.358889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c87f1-cddd-4cc0-9f8c-5a69ca402802 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:52.373815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 806/1610] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:52.375757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65845fd-4043-45fb-9ac6-cb9e8d3a0e6f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.376353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65845fd-4043-45fb-9ac6-cb9e8d3a0e6f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.376681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65845fd-4043-45fb-9ac6-cb9e8d3a0e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.379842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f7b8-f460-47ee-9835-452aefde4593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=373eb8dd7c4447bf983907b3e73014a1, user_id=9f4bff49703c4c549ece69fc45e3cdc9, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:52.383829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f7b8-f460-47ee-9835-452aefde4593 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:52.402889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a65845fd-4043-45fb-9ac6-cb9e8d3a0e6f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:52.405344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a65845fd-4043-45fb-9ac6-cb9e8d3a0e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:52.407107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 805/1611] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:52 2022] PUT /identity/v3/projects/373eb8dd7c4447bf983907b3e73014a1/users/9f4bff49703c4c549ece69fc45e3cdc9/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:52.408881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da1b17-8b09-42d9-9b80-ba921eef8359 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.409593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da1b17-8b09-42d9-9b80-ba921eef8359 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.410248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da1b17-8b09-42d9-9b80-ba921eef8359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.471649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-80da1b17-8b09-42d9-9b80-ba921eef8359 None None] MFA Rules not processed for user `6173cb93d6214493b75a91f2ae9e8a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:52.473481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-80da1b17-8b09-42d9-9b80-ba921eef8359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:52.555502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 807/1612] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:52.555502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7bd54-8a72-4cac-93bc-63ad1c0f429a None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.555502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7bd54-8a72-4cac-93bc-63ad1c0f429a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.555502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7bd54-8a72-4cac-93bc-63ad1c0f429a None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.555502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 808/1613] 10.222.0.72 () {54 vars in 961 bytes} [Mon Sep 26 17:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:58:52.555502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7bd54-8a72-4cac-93bc-63ad1c0f429a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.593168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 806/1614] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:52.593168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.593168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.593168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.631477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] MFA Rules not processed for user `9f4bff49703c4c549ece69fc45e3cdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:52.632877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:52.687336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7bd54-8a72-4cac-93bc-63ad1c0f429a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:52.693086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d4e4b-1301-4ac4-961b-8f497142ed40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:52.694093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d4e4b-1301-4ac4-961b-8f497142ed40 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.694409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d4e4b-1301-4ac4-961b-8f497142ed40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.694944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d4e4b-1301-4ac4-961b-8f497142ed40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.708465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d4e4b-1301-4ac4-961b-8f497142ed40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:52.710891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d4e4b-1301-4ac4-961b-8f497142ed40 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:52.726212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 809/1615] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:52.727954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.728288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.728568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.730737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:52.749640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 807/1616] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:52.752094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.766822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f924-5ddf-4d64-9b11-94d8df0e734b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:52.769726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:52.780139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:52.781434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:52.782904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:52.783393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:52.784574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:52.788168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:52.826313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:52.831881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:52.835253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 808/1617] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:52.836391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.971756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 810/1618] 10.222.0.72 () {58 vars in 1058 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:52.972742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:52.989696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54225c54-c9cc-4798-9bbc-b9b582695556 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:52.999059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.001303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.002188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.003014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.067615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e3894-5ec1-4dd3-bad4-3d58c0a95584 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.075119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.075790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.077058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.077058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.079271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.081884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.137131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 811/1619] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:58:52 2022] POST /identity/v3/projects => generated 403 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:53.137131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.167337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.168619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.206574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 809/1620] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:53.207637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.237184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e3aa6-0dd0-41e8-8d41-3256ac0be17c service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.295759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc4da-a6fb-4d0f-aa94-5898c0a90c69 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.310439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.311397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.313160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.314471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.317755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.324537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.363118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:53.377353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.378818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.378998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 812/1621] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:53 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:53.379496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.379886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.380616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.415847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f053-2a14-462c-9bbf-6ca6ead1ced6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.429794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:53.436231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.437234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.439091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.439572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.440725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.450294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.507725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 813/1622] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:58:53 2022] POST /identity/v3/users => generated 308 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:58:53.509779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.535060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402092f-94bb-460d-9836-d912aa9bf23a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.541560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.544264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.553824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.554786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.555098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.555378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.556039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.557663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.559547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 810/1623] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:53.560622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.583174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7faf86-3722-4665-b24f-b956ab6b017b service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.592292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.593021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.594397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.594397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.594774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.596589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.612526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 811/1624] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:53.612526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.617362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 814/1625] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:53.619699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.622871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c126c7-c0fe-4d2b-ba7c-5a9c8178c4f0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.628126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.628928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.629223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.629624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] PATH_INFO: `/v3/projects/cc4114a9ef274f51a518428a68fb64cb/users/c1426375453f4b55bdc5bdd7501f9a08/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.656112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369908ff-b18b-4823-8128-4768a46f6254 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.663451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc4114a9ef274f51a518428a68fb64cb, user_id=c1426375453f4b55bdc5bdd7501f9a08, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.666800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.675609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.676295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.676578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.676861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] PATH_INFO: `/v3/projects/811ba71a6100434896eb0bba550acafb/users/63e4f29bf82c4f32a57de1f06c28ad03/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.686320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 812/1626] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:53 2022] PUT /identity/v3/projects/cc4114a9ef274f51a518428a68fb64cb/users/c1426375453f4b55bdc5bdd7501f9a08/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:58:53.687625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.730137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811ba71a6100434896eb0bba550acafb, user_id=63e4f29bf82c4f32a57de1f06c28ad03, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.736785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.764023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 815/1627] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:53 2022] PUT /identity/v3/projects/811ba71a6100434896eb0bba550acafb/users/63e4f29bf82c4f32a57de1f06c28ad03/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:53.768751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.838627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb9cf-d23c-4b6b-a4c1-9e698286425b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.906057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d457f-bcc2-47a2-8d98-1dd7f92030f4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.913814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.914589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.915424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.915806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.916572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.918329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:53.939924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 816/1628] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:53.943478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:53.956930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e522d-dd6b-4486-8627-026b7db317cd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:53.963295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce84f8-e209-4c28-9253-15870a5715a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.964143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce84f8-e209-4c28-9253-15870a5715a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.965041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce84f8-e209-4c28-9253-15870a5715a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.965428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce84f8-e209-4c28-9253-15870a5715a5 admin admin] PATH_INFO: `/v3/projects/811ba71a6100434896eb0bba550acafb/users/63e4f29bf82c4f32a57de1f06c28ad03/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.976262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce84f8-e209-4c28-9253-15870a5715a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811ba71a6100434896eb0bba550acafb, user_id=63e4f29bf82c4f32a57de1f06c28ad03, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.976768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:53.982867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:53.983212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:53.983577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:53.984282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:53.986085 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:54.000883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 813/1629] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:58:54.002427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce84f8-e209-4c28-9253-15870a5715a5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:54.006023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:54.030906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 817/1630] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:53 2022] PUT /identity/v3/projects/811ba71a6100434896eb0bba550acafb/users/63e4f29bf82c4f32a57de1f06c28ad03/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:54.032837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce9c9a-866e-4091-a2d4-ffbc660e0ad1 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.033185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce9c9a-866e-4091-a2d4-ffbc660e0ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.033525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce9c9a-866e-4091-a2d4-ffbc660e0ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.037082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad95a4-c42d-47b1-ab61-c98127d5d568 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:54.070893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-1cce9c9a-866e-4091-a2d4-ffbc660e0ad1 None None] MFA Rules not processed for user `63e4f29bf82c4f32a57de1f06c28ad03`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:54.073479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1cce9c9a-866e-4091-a2d4-ffbc660e0ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:54.114310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:54.181902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940f79a-694b-4d30-a32e-e45d6f0897f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:54.182592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940f79a-694b-4d30-a32e-e45d6f0897f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.182908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940f79a-694b-4d30-a32e-e45d6f0897f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.183185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940f79a-694b-4d30-a32e-e45d6f0897f9 admin admin] PATH_INFO: `/v3/projects/cc4114a9ef274f51a518428a68fb64cb/users/c1426375453f4b55bdc5bdd7501f9a08/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.185160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940f79a-694b-4d30-a32e-e45d6f0897f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc4114a9ef274f51a518428a68fb64cb, user_id=c1426375453f4b55bdc5bdd7501f9a08, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:54.189488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940f79a-694b-4d30-a32e-e45d6f0897f9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:54.212299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 814/1631] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:58:54 2022] PUT /identity/v3/projects/cc4114a9ef274f51a518428a68fb64cb/users/c1426375453f4b55bdc5bdd7501f9a08/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:54.213888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198eaf-73f3-405b-a70b-61715c0e47a7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.214229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198eaf-73f3-405b-a70b-61715c0e47a7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.214508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198eaf-73f3-405b-a70b-61715c0e47a7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.221208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 815/1632] 10.222.0.72 () {54 vars in 912 bytes} [Mon Sep 26 17:58:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:58:54.223049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.223344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.223670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.255301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 818/1633] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:54.263061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] MFA Rules not processed for user `c1426375453f4b55bdc5bdd7501f9a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:54.264498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:54.264968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.265268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.267750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.317799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:54.319894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:54.352512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:54.360743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 816/1634] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:54.364263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:54.466538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4dc28-e668-401d-9e28-f1769cbdfc18 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:54.478879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:54.480336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.482069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.482362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.523735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 819/1635] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 17:58:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:54.525273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:54.552426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226e9e8-0eab-4de0-b92c-3f52883706c4 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:54.563836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:54.564468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.566554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.566863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.595678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:54.597190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:54.634268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:54.636027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:54.637892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 820/1636] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:54.642140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:54.650279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 817/1637] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:54.746983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:54.769110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c774908-e39a-4e5d-a267-da7c6334732f service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:54.792785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:54.799059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.799400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.799640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:54.831498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:54.833064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:54.861661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 821/1638] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:54.951661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:54.965947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c13ac3-0792-43b8-b2ca-5ab0efc1fcda service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:54.974913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c94501-aab7-4095-a494-e27fe12d525e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:54.975930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c94501-aab7-4095-a494-e27fe12d525e service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:54.976316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c94501-aab7-4095-a494-e27fe12d525e service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:54.976816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c94501-aab7-4095-a494-e27fe12d525e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:55.097023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c94501-aab7-4095-a494-e27fe12d525e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:55.103584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c94501-aab7-4095-a494-e27fe12d525e service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:55.127995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 818/1639] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:55.669000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:55.870814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1e951-feba-4481-bb23-80e3c4c7b01d service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:55.880920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:55.881746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:55.882264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:55.882667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] PATH_INFO: `/v3/users/9f4bff49703c4c549ece69fc45e3cdc9` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:55.884409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4bff49703c4c549ece69fc45e3cdc9)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:55.886430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:55.998740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4bff49703c4c549ece69fc45e3cdc9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:56.000563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4bff49703c4c549ece69fc45e3cdc9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:56.020118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4bff49703c4c549ece69fc45e3cdc9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:56.064212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4bff49703c4c549ece69fc45e3cdc9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:56.065492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4bff49703c4c549ece69fc45e3cdc9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:56.088989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:56.111855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 822/1640] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:58:55 2022] DELETE /identity/v3/users/9f4bff49703c4c549ece69fc45e3cdc9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:56.117793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd83283-c74a-4a9c-9bee-ff5a55aca7b1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:56.119260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd83283-c74a-4a9c-9bee-ff5a55aca7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:56.121273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd83283-c74a-4a9c-9bee-ff5a55aca7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:56.153692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8bd83283-c74a-4a9c-9bee-ff5a55aca7b1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:56.156274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8bd83283-c74a-4a9c-9bee-ff5a55aca7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:56.258199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 819/1641] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:56.291481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:56.294462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:56.484150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b06b4-22a0-4da7-b4a4-8f7931723106 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:56.498731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611d2ad-6a0c-45aa-9667-16d1a0b15911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:56.507978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611d2ad-6a0c-45aa-9667-16d1a0b15911 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:56.509944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611d2ad-6a0c-45aa-9667-16d1a0b15911 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:56.510536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611d2ad-6a0c-45aa-9667-16d1a0b15911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:56.540010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611d2ad-6a0c-45aa-9667-16d1a0b15911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:56.545977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611d2ad-6a0c-45aa-9667-16d1a0b15911 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:56.570884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 823/1642] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:57.763795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27268d-a438-45d9-a9b8-9d0363893abc None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:57.764313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27268d-a438-45d9-a9b8-9d0363893abc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:57.764389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27268d-a438-45d9-a9b8-9d0363893abc None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:57.767817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 820/1643] 10.222.0.72 () {54 vars in 961 bytes} [Mon Sep 26 17:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:58:57.793757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf2e4c-b18f-4fa1-9f25-132544b5aa7e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:57.794286 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf2e4c-b18f-4fa1-9f25-132544b5aa7e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:57.795190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf2e4c-b18f-4fa1-9f25-132544b5aa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:57.857383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3acf2e4c-b18f-4fa1-9f25-132544b5aa7e None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:57.864630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3acf2e4c-b18f-4fa1-9f25-132544b5aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:58.069418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 824/1644] 10.222.0.72 () {58 vars in 1058 bytes} [Mon Sep 26 17:58:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:58.075757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0fe8-99dd-42de-a36a-6dd849eeed44 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:58.076045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0fe8-99dd-42de-a36a-6dd849eeed44 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:58.076325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0fe8-99dd-42de-a36a-6dd849eeed44 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:58.077917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 821/1645] 10.222.0.72 () {54 vars in 912 bytes} [Mon Sep 26 17:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:58:58.083769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:58.084098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:58.084578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:58.128905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:58:58.130516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:58.147769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0fe8-99dd-42de-a36a-6dd849eeed44 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:58.149646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d4db0fe8-99dd-42de-a36a-6dd849eeed44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:58.151642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 825/1646] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 17:58:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:58:58.175636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:58.188381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:58:58.216536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0fe8-99dd-42de-a36a-6dd849eeed44 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:58.230814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:58.233522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:58.234753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:58.236687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] PATH_INFO: `/v3/projects/373eb8dd7c4447bf983907b3e73014a1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:58.239932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=373eb8dd7c4447bf983907b3e73014a1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:58.251750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:58.360868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea49a94-8b59-4dc8-8d44-66ce95611a9b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:58.366591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:58.367313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:58.367715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:58.372605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:58.411539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373eb8dd7c4447bf983907b3e73014a1', 'request_id': 'req-53f34556-c3cf-495d-9a1e-8bda33096d11'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.413334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:58.415322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:58.422193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373eb8dd7c4447bf983907b3e73014a1', 'request_id': 'req-53f34556-c3cf-495d-9a1e-8bda33096d11'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.450341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 826/1647] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:58:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:58.473993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373eb8dd7c4447bf983907b3e73014a1', 'request_id': 'req-53f34556-c3cf-495d-9a1e-8bda33096d11'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.508546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] The token cache is being invalidate because project 373eb8dd7c4447bf983907b3e73014a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:58:58.513872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.515336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.517552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 822/1648] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:58:58 2022] DELETE /identity/v3/projects/373eb8dd7c4447bf983907b3e73014a1 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:58:58.521640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:58:58.539186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db6411-d662-46b6-ac27-c965daf6ae96 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:58:58.612297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:58.729452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:58:58.730126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:58:58.730408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:58:58.730682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] PATH_INFO: `/v3/users/63e4f29bf82c4f32a57de1f06c28ad03` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:58:58.732384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e4f29bf82c4f32a57de1f06c28ad03)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:58:58.739975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:58:58.890452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e4f29bf82c4f32a57de1f06c28ad03'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.891674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e4f29bf82c4f32a57de1f06c28ad03'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.917650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e4f29bf82c4f32a57de1f06c28ad03'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.953439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e4f29bf82c4f32a57de1f06c28ad03'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.959418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3ce8366a-570f-4291-b246-67dc3a20893e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e4f29bf82c4f32a57de1f06c28ad03'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:58:58.993197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 827/1649] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:58:58 2022] DELETE /identity/v3/users/63e4f29bf82c4f32a57de1f06c28ad03 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:00.870070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:00.873183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:00.909226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34556-c3cf-495d-9a1e-8bda33096d11 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:00.934618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:01.017283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:01.018099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.018481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.018925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] PATH_INFO: `/v3/projects/811ba71a6100434896eb0bba550acafb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.020288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=811ba71a6100434896eb0bba550acafb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:01.022869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:01.092447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9062b5-830b-45ab-ad89-320b5fec786d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.093006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9062b5-830b-45ab-ad89-320b5fec786d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.094821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9062b5-830b-45ab-ad89-320b5fec786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.177940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-da9062b5-830b-45ab-ad89-320b5fec786d None None] MFA Rules not processed for user `008e6c2c897741889510e6bf5181a9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:01.186356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-da9062b5-830b-45ab-ad89-320b5fec786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:01.207586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811ba71a6100434896eb0bba550acafb', 'request_id': 'req-1b4031f7-41f5-4046-94cc-3be3b32ab11a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:01.230402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811ba71a6100434896eb0bba550acafb', 'request_id': 'req-1b4031f7-41f5-4046-94cc-3be3b32ab11a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:01.245512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811ba71a6100434896eb0bba550acafb', 'request_id': 'req-1b4031f7-41f5-4046-94cc-3be3b32ab11a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:01.272011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] The token cache is being invalidate because project 811ba71a6100434896eb0bba550acafb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:01.272458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:01.275653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1b4031f7-41f5-4046-94cc-3be3b32ab11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:01.281156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 823/1650] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:00 2022] DELETE /identity/v3/projects/811ba71a6100434896eb0bba550acafb => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:01.283523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce74fd-48ab-4b6f-980e-6ecec362293c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.283918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce74fd-48ab-4b6f-980e-6ecec362293c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.284315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce74fd-48ab-4b6f-980e-6ecec362293c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.327003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e4ce74fd-48ab-4b6f-980e-6ecec362293c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:01.335046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e4ce74fd-48ab-4b6f-980e-6ecec362293c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:01.350051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 828/1651] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:01.357090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.357518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.357901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.408953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] MFA Rules not processed for user `008e6c2c897741889510e6bf5181a9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:01.415073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:01.434466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 829/1652] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:01.446251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:01.449168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:01.518287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 824/1653] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:01.527630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.528021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.528399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.543465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:01.582582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:01.584611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:01.605039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 825/1654] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:01.607961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:01.626572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324229b-cbc2-4072-a5ce-7c988b215851 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:01.628461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d4af6-5198-4cc2-89e9-8242adf6a359 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:01.637136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:01.638381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.639256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.640061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.641420 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:01.642515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:01.643311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:01.644066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.645824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:01.646221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.646606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.662255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:01.665596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:01.674795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 826/1655] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:01.678759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:01.686522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 830/1656] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:01.712736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef2954-87dd-4cdb-99f4-f4030114b71e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:01.724468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:01.726633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:01.726633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:01.726633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:01.726633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:01.731367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:01.792857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 827/1657] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:01 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:01.796825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:01.815473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1377db-9f39-41a5-8d4a-d8a73d6bc700 service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:02.045536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:02.046609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.047576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.048075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.048962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:02.051273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:02.096713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 831/1658] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:01 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:02.101495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:02.117713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16454acd-6eb4-4a2d-8e8e-77c4dcd0b02a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:02.130044 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:02.131048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.131571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.132069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.132931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:02.134778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:02.144496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:02.172292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 828/1659] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:02.174038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:02.189157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0350-ad2f-48f2-9ec0-a6a297f4a7a6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:02.202940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea282-bffc-47e2-a6ad-557ea1548f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:02.204586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea282-bffc-47e2-a6ad-557ea1548f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.205088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea282-bffc-47e2-a6ad-557ea1548f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.205609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea282-bffc-47e2-a6ad-557ea1548f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.206474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea282-bffc-47e2-a6ad-557ea1548f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:02.210065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea282-bffc-47e2-a6ad-557ea1548f20 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:02.233585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 832/1660] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:02.237895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:02.251393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b070-fc47-46dc-82f7-7fc842300fba admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:02.261427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:02.263889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.264297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.264746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] PATH_INFO: `/v3/projects/8ea76e6071a9473f88a3a93981018bea/users/b024f9a70fd848308d6b97365a536c36/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.302740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea76e6071a9473f88a3a93981018bea, user_id=b024f9a70fd848308d6b97365a536c36, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:02.305657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:02.343166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fdd3c-e362-4ec6-8b4c-d62fec2402f7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.343669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fdd3c-e362-4ec6-8b4c-d62fec2402f7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.344206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fdd3c-e362-4ec6-8b4c-d62fec2402f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.345329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 829/1661] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:02 2022] PUT /identity/v3/projects/8ea76e6071a9473f88a3a93981018bea/users/b024f9a70fd848308d6b97365a536c36/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:02.406529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a21fdd3c-e362-4ec6-8b4c-d62fec2402f7 None None] MFA Rules not processed for user `b024f9a70fd848308d6b97365a536c36`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:02.407818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a21fdd3c-e362-4ec6-8b4c-d62fec2402f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:02.580682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 833/1662] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:02.586729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:02.598412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afd65-1dc4-4abe-b44c-4d6ebfbf8ed8 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.598705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afd65-1dc4-4abe-b44c-4d6ebfbf8ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.599032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afd65-1dc4-4abe-b44c-4d6ebfbf8ed8 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.600677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 834/1663] 10.222.0.72 () {54 vars in 912 bytes} [Mon Sep 26 17:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:02.617352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.617910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.618036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.758116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:02.760378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:02.793219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:02.815162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58d272-ddbb-4e82-895e-a5b3d8e6978d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:02.823951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:02.824610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:02.824896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:02.825174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:02.826430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:02.833887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:02.932328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 830/1664] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:02 2022] POST /identity/v3/projects => generated 389 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:02.947646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.008941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 835/1665] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 17:59:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:03.038947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.040435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:03.055163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3c601-7195-45be-983c-f523dd85fd87 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:03.146370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ea58b-c83d-461b-b9b5-be6102025897 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:03.157927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:03.158611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.160913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.161214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.163431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:03.168494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:03.198246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:03.199652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.201612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.202894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.235215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 831/1666] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:02 2022] POST /identity/v3/users => generated 301 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:03.237904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:03.243310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.247583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:03.277958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf4dff-684c-4a39-a9df-cd915cb604e2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:03.279010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 836/1667] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:03.285851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:03.286593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.287054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.287335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.288042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:03.289767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:03.307779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.326428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 832/1668] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:03.336104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91634-8892-448b-b2b2-08d13794dae5 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:03.345435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:03.346285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.346676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.347091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] PATH_INFO: `/v3/projects/24beedb157674785a4dad862cf1cb16a/users/8b44011a95ad456880d9a44d14e56f9d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.364690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] RBAC: Authorizing `identity:create_grant(project_id=24beedb157674785a4dad862cf1cb16a, user_id=8b44011a95ad456880d9a44d14e56f9d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:03.367982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:03.392823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 837/1669] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:03 2022] PUT /identity/v3/projects/24beedb157674785a4dad862cf1cb16a/users/8b44011a95ad456880d9a44d14e56f9d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:03.397100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.413907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d178eca-dfec-4d2a-a2da-573ce41b6fd4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:03.502318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:03.559064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c46e2-f67c-4e3c-994d-cb965b59bca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:03.559891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c46e2-f67c-4e3c-994d-cb965b59bca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.560180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c46e2-f67c-4e3c-994d-cb965b59bca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.560456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c46e2-f67c-4e3c-994d-cb965b59bca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.561242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c46e2-f67c-4e3c-994d-cb965b59bca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:03.564687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c46e2-f67c-4e3c-994d-cb965b59bca2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:03.598970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 833/1670] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:03.602878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.617810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa25e40-6b07-4e61-b1ae-e1bc6ad8087f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:03.639538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:03.640233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.641051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.641375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] PATH_INFO: `/v3/projects/24beedb157674785a4dad862cf1cb16a/users/8b44011a95ad456880d9a44d14e56f9d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.649222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24beedb157674785a4dad862cf1cb16a, user_id=8b44011a95ad456880d9a44d14e56f9d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:03.659568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:03.693972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:03.694323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:03.694535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:03.697486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 838/1671] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:03 2022] PUT /identity/v3/projects/24beedb157674785a4dad862cf1cb16a/users/8b44011a95ad456880d9a44d14e56f9d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:03.737457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] MFA Rules not processed for user `8b44011a95ad456880d9a44d14e56f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:03.738870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:03.844938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 834/1672] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:03 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:03.860030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:03.866530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:04.045053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa988e7-f48e-4458-a1a4-98ae5c61ff28 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:04.059260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:04.060186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:04.060632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:04.061061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:04.095641 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:04.097464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:04.134683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 839/1673] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:04.425626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:04.493161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:04.494667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:04.577452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17e04f-b73b-4f80-957c-9b4d61d7bfa9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:04.584289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:04.584952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:04.585237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:04.585515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] PATH_INFO: `/v3/users/b024f9a70fd848308d6b97365a536c36` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:04.587025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b024f9a70fd848308d6b97365a536c36)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:04.588119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: WARNING py.warnings [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:04.588119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: warnings.warn(msg) Sep 26 17:59:04.588119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: Sep 26 17:59:04.590077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:04.599707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:04.652260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32060e-5a35-423b-b3c6-94c1c5c38840 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:04.668417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:04.669261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:04.673556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:04.673556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:04.675476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:04.716468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:04.718083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63928066-26d0-4b03-b5ee-73da578f1397 service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:04.721880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b024f9a70fd848308d6b97365a536c36'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:04.723115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b024f9a70fd848308d6b97365a536c36'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:04.725316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b024f9a70fd848308d6b97365a536c36'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:04.745771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 840/1674] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:59:04.751060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b024f9a70fd848308d6b97365a536c36'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:04.767951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:04.780475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:04.780835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:04.781182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:04.786879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9d8435c2-03ed-413e-bee4-12eabe3f697b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b024f9a70fd848308d6b97365a536c36'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:04.818962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 835/1675] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:04 2022] DELETE /identity/v3/users/b024f9a70fd848308d6b97365a536c36 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:04.824178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] MFA Rules not processed for user `008e6c2c897741889510e6bf5181a9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:04.825448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:04.827435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23607da6-4e47-4d80-b91b-d4d4b2cd2c8a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:04.827834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23607da6-4e47-4d80-b91b-d4d4b2cd2c8a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:04.828906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23607da6-4e47-4d80-b91b-d4d4b2cd2c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:04.863105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-23607da6-4e47-4d80-b91b-d4d4b2cd2c8a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:04.866504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-23607da6-4e47-4d80-b91b-d4d4b2cd2c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:05.019372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 841/1676] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:05.091309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 836/1677] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:05.257186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:05.259485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:05.360282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced01f3-f513-4d02-beee-2e7d9162b68e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:05.369461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:05.370390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:05.370797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:05.371244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:05.389514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:05.391857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:05.418452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 842/1678] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:06.908306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:06.909399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:06.909907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:06.976588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] MFA Rules not processed for user `4bbd5a38a6ce472790128a37f62d472b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:06.978703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:07.197653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 837/1679] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:07.229430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:07.231415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:07.346951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89466a28-1802-400e-962e-e43abdf0a520 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:07.352931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:07.353752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:07.354040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:07.354310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:07.361758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:07.366560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:07.368213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:07.383825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 843/1680] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:59:08.020747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:08.022227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:08.056410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2612f-6b97-431b-abb6-9f40611afa4d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:08.077327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:08.081274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:08.081570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:08.081849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] PATH_INFO: `/v3/projects/8ea76e6071a9473f88a3a93981018bea` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:08.086442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea76e6071a9473f88a3a93981018bea)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:08.093854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:08.238494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea76e6071a9473f88a3a93981018bea', 'request_id': 'req-70163280-df79-4efb-85ab-295b4faadf75'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.260725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea76e6071a9473f88a3a93981018bea', 'request_id': 'req-70163280-df79-4efb-85ab-295b4faadf75'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.296201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:08.300064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:08.302175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea76e6071a9473f88a3a93981018bea', 'request_id': 'req-70163280-df79-4efb-85ab-295b4faadf75'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.322227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:08.346348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3476d7d-ec06-4cb0-97f2-2436d3ed8d1c service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:08.365113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] The token cache is being invalidate because project 8ea76e6071a9473f88a3a93981018bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:08.367330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.369243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.380739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 838/1681] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:08 2022] DELETE /identity/v3/projects/8ea76e6071a9473f88a3a93981018bea => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:08.385859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:08.391876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:08.596594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:08.601753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:08.602076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:08.602363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] PATH_INFO: `/v3/users/d667778c6e7b4879bc6c65dc82e230d1` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:08.615649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d667778c6e7b4879bc6c65dc82e230d1)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:08.618439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:08.642450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70163280-df79-4efb-85ab-295b4faadf75 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:08.664892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:08.667617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:08.668135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:08.669399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] PATH_INFO: `/v3/users/8b44011a95ad456880d9a44d14e56f9d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:08.672426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b44011a95ad456880d9a44d14e56f9d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:08.683228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:08.806553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd667778c6e7b4879bc6c65dc82e230d1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.810944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd667778c6e7b4879bc6c65dc82e230d1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.830008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b44011a95ad456880d9a44d14e56f9d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.832489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b44011a95ad456880d9a44d14e56f9d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.833410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b44011a95ad456880d9a44d14e56f9d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.836805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd667778c6e7b4879bc6c65dc82e230d1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.868303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b44011a95ad456880d9a44d14e56f9d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.884181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd667778c6e7b4879bc6c65dc82e230d1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.886766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-257d9ad6-cd56-4c6f-9bd2-120c53e5e0a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd667778c6e7b4879bc6c65dc82e230d1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.915262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 844/1682] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:08 2022] DELETE /identity/v3/users/d667778c6e7b4879bc6c65dc82e230d1 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:08.922705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf817ce3-f564-40a7-be31-8e311155415c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:08.923303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf817ce3-f564-40a7-be31-8e311155415c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:08.923731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf817ce3-f564-40a7-be31-8e311155415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:08.953598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b44011a95ad456880d9a44d14e56f9d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:08.985773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cf817ce3-f564-40a7-be31-8e311155415c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:08.987357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cf817ce3-f564-40a7-be31-8e311155415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:09.019775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 839/1683] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:08 2022] DELETE /identity/v3/users/8b44011a95ad456880d9a44d14e56f9d => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:09.153673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 845/1684] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:09.349417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:09.353504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:09.524327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:09.525326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:09.525841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:09.600892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079a45c-4b0d-4731-bdd8-7b5ab5b5e87f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:09.605601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:09.630244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:09.631259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:09.634240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:09.635543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:09.675637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:09.677502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:09.754379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:09.863624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 846/1685] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 17:59:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:09.895159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:09.896636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:09.925220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 840/1686] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:10.144014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:10.146693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:10.490722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209b4ef-0df6-4e24-8bc4-dab8ad18de73 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:10.497024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6ac0-1a13-4b56-87c7-d04277b55973 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:10.497681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6ac0-1a13-4b56-87c7-d04277b55973 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:10.503849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6ac0-1a13-4b56-87c7-d04277b55973 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:10.506776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6ac0-1a13-4b56-87c7-d04277b55973 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:10.531473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b6ac0-1a13-4b56-87c7-d04277b55973 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:10.535545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b6ac0-1a13-4b56-87c7-d04277b55973 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:10.561190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 847/1687] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 17:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 17:59:11.128537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:11.130115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:11.159592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73512631-9815-4ba7-86cc-6b11d68ebde6 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:11.173405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:11.174226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:11.174760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:11.175280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] PATH_INFO: `/v3/users/6173cb93d6214493b75a91f2ae9e8a1f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:11.177954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6173cb93d6214493b75a91f2ae9e8a1f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:11.180263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:11.332517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6173cb93d6214493b75a91f2ae9e8a1f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:11.335251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6173cb93d6214493b75a91f2ae9e8a1f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:11.336829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6173cb93d6214493b75a91f2ae9e8a1f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:11.368166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6173cb93d6214493b75a91f2ae9e8a1f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:11.411519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6173cb93d6214493b75a91f2ae9e8a1f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:11.447105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 841/1688] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:11 2022] DELETE /identity/v3/users/6173cb93d6214493b75a91f2ae9e8a1f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:11.451615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481be0f-456b-4635-b082-d627a845109c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:11.452386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481be0f-456b-4635-b082-d627a845109c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:11.452907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481be0f-456b-4635-b082-d627a845109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:11.476302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6481be0f-456b-4635-b082-d627a845109c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:11.477948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6481be0f-456b-4635-b082-d627a845109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:11.551659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:11.585898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 848/1689] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:11.817598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:11.819073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:12.006209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d89757-e9ac-45dd-98c3-1df71541edef admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:12.017454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:12.018265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:12.019598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:12.019900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:12.040713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:12.042201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:12.076096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 842/1690] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:12.681676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:12.682668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:12.683187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:12.749434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] MFA Rules not processed for user `276174ef02b1409b840a0de13aa4095d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:12.751722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:12.954000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:12.957224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:12.960412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 849/1691] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:12.971805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:12.974157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:12.981019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e17eff-6419-44b4-a74e-2ac382251751 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:12.989208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:12.990067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:12.990489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:12.990965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] PATH_INFO: `/v3/projects/24beedb157674785a4dad862cf1cb16a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:12.992463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24beedb157674785a4dad862cf1cb16a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:12.997618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:13.112523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:13.200019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24beedb157674785a4dad862cf1cb16a', 'request_id': 'req-8e3ab70a-bfee-449b-820b-b2d563f9b908'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:13.247415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24beedb157674785a4dad862cf1cb16a', 'request_id': 'req-8e3ab70a-bfee-449b-820b-b2d563f9b908'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:13.279706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24beedb157674785a4dad862cf1cb16a', 'request_id': 'req-8e3ab70a-bfee-449b-820b-b2d563f9b908'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:13.338034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c940d-1b74-46f8-a8b5-c80e0ac9bf16 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:13.339468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] The token cache is being invalidate because project 24beedb157674785a4dad862cf1cb16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:13.341708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:13.342829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8e3ab70a-bfee-449b-820b-b2d563f9b908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:13.350753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 843/1692] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:12 2022] DELETE /identity/v3/projects/24beedb157674785a4dad862cf1cb16a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:13.352555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b108-6970-4a89-b13f-e1e6a8854483 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:13.352945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b108-6970-4a89-b13f-e1e6a8854483 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:13.353890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b108-6970-4a89-b13f-e1e6a8854483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:13.412574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9b30b108-6970-4a89-b13f-e1e6a8854483 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:13.414130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9b30b108-6970-4a89-b13f-e1e6a8854483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:13.602943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:13.606356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:13.610125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:13.611332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:13.617883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:13.646052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 844/1693] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:13.654201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:13.654735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:13.655273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:13.706187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:13.711395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:13.733705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:13.737445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 845/1694] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:13.742505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:13.785167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feeac25-f94d-4519-812e-66aff0485f38 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:13.799327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:13.801189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:13.801484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:13.801770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:13.803712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:13.805213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:13.810210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:13.863863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 846/1695] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:13.869275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:13.874825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:13.877395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:13.917514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46cfbf-0a5c-4842-a7b7-bf6efe220082 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:13.921359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 850/1696] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 950 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:13.932545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:13.933174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:13.933458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:13.937928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:13.944346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:13.949504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:13.950680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:13.952329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:13.991444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b3fba-e833-42b7-bd72-6d93eedaed8b service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:14.006456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:14.014998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.018655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.019615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] PATH_INFO: `/v3/projects/41f7e2a3f28f4734ae623883e0c53aeb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.025425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f7e2a3f28f4734ae623883e0c53aeb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:14.026968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 847/1697] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:13 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:14.029351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: WARNING py.warnings [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:14.029351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: warnings.warn(msg) Sep 26 17:59:14.029351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: Sep 26 17:59:14.032693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:14.035296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:14.191678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bdcfd-d24e-4e4f-8042-5acb39c4d162 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:14.253789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:14.266277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f7e2a3f28f4734ae623883e0c53aeb', 'request_id': 'req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:14.296048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f7e2a3f28f4734ae623883e0c53aeb', 'request_id': 'req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:14.344488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f7e2a3f28f4734ae623883e0c53aeb', 'request_id': 'req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:14.350386 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da108664-57fb-440a-b300-c8be80441837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:14.351291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da108664-57fb-440a-b300-c8be80441837 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.352615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da108664-57fb-440a-b300-c8be80441837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.353196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da108664-57fb-440a-b300-c8be80441837 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.354081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da108664-57fb-440a-b300-c8be80441837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:14.358292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da108664-57fb-440a-b300-c8be80441837 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:14.380774 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] The token cache is being invalidate because project 41f7e2a3f28f4734ae623883e0c53aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:14.381092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:14.384153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-bad672e5-56d3-4ea7-90fc-a9e8b1d16b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:14.391943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 851/1698] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:13 2022] DELETE /identity/v3/projects/41f7e2a3f28f4734ae623883e0c53aeb => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:14.393753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.394195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.394581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.444536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 848/1699] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:14 2022] POST /identity/v3/users => generated 293 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:14.446116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.446443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.446735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.537143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:14.538635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:14.558227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 852/1700] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:14.558227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:14.561264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:14.561751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] MFA Rules not processed for user `28195a63990344edb6ea31f1f4463a82`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:14.563935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:14.593171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1119-1186-4cc1-9c4e-87ea02e24882 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:14.616623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104ba4d-1111-466d-bc63-d9a9aa6a79e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:14.617259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104ba4d-1111-466d-bc63-d9a9aa6a79e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.617582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104ba4d-1111-466d-bc63-d9a9aa6a79e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.617855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104ba4d-1111-466d-bc63-d9a9aa6a79e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.618503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104ba4d-1111-466d-bc63-d9a9aa6a79e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:14.620440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104ba4d-1111-466d-bc63-d9a9aa6a79e6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:14.653796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 853/1701] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:14.656030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.656348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.656787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.727576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:14.732853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:14.778130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 854/1702] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:14.778848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:14.796030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d823d-aafe-44ec-85f6-a2e006107770 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:14.815493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 849/1703] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:14.815493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:14.818729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:14.819394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.819672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.820541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.820665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:14.823012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:14.837771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 855/1704] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:14.838814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:14.842205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:14.854324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb5f50-4f14-4a82-85e9-9b2dbe3d4b80 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:14.869913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:14.870533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.871517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.871805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:14.872478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:14.872781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:14.874416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:14.912384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 850/1705] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:14.913340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:14.932884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3455d43-7db5-4728-aaec-1cbbdbb41063 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:14.953418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:14.955355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:14.955898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:14.956408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] PATH_INFO: `/v3/projects/44ec15b901f942cb9a9cc57541cfb201/users/8c0e8ff68e1847c99ba125c38ac0450e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.009803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ec15b901f942cb9a9cc57541cfb201, user_id=8c0e8ff68e1847c99ba125c38ac0450e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.028149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.045384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c36c0-930a-480a-995e-947b99a4b071 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.056319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47cf04-1854-4c14-bd52-05c5bcb49cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.057167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47cf04-1854-4c14-bd52-05c5bcb49cb7 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.057454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47cf04-1854-4c14-bd52-05c5bcb49cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.057737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47cf04-1854-4c14-bd52-05c5bcb49cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.063756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:15.069058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47cf04-1854-4c14-bd52-05c5bcb49cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.071581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47cf04-1854-4c14-bd52-05c5bcb49cb7 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.090276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 851/1706] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:14 2022] PUT /identity/v3/projects/44ec15b901f942cb9a9cc57541cfb201/users/8c0e8ff68e1847c99ba125c38ac0450e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:15.091310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.107547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 856/1707] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:15.109123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.110006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.110286 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.220112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] MFA Rules not processed for user `8c0e8ff68e1847c99ba125c38ac0450e`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:15.221371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:15.342373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d58498-0c8f-4240-b23b-0d1891cf0b8b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.366004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 857/1708] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:15.367172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.372244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.372885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.373168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.373444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.374093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.378894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.388743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb345a-aa7d-4275-a442-d5441828dbb7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.412335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.412971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.413249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.413520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.414158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.416105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.417706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:15.464724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 852/1709] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:15 2022] POST /identity/v3/projects => generated 365 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:15.471396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.491151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 858/1710] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:15 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:15.494632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.665225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c64b-22d6-4565-a076-6e2c215f93e3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.685634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f9009-dfea-4c04-a84b-fb7b97121758 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.689273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.690010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.690278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.690546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.691243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.699950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.701984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.707478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.707478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.707478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.709604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.715984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.762805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 859/1711] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:15 2022] POST /identity/v3/users => generated 300 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:15.767086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.788189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba45c3e-0c22-4c1f-b6b1-6e26e815c037 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.817477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.818108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.818382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.818649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.819500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.821092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.827454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 853/1712] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:15 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:15.831092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.851646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 860/1713] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:15.855496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.869444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e6323-f9b1-452d-9768-4aba5967e752 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.880619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23f58c-483a-4f8d-8166-524a32915a2f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:15.897245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.898418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.898854 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.899275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] PATH_INFO: `/v3/projects/83c74e06abe8467fae76793f536ae17a/users/fed6b7efe95141cf875944e41639f664/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.899632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:15.900271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:15.900552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:15.900833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:15.901487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.903179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.932012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c74e06abe8467fae76793f536ae17a, user_id=fed6b7efe95141cf875944e41639f664, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:15.934782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:15.962356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 854/1714] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:15.964153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:15.982945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 861/1715] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:15 2022] PUT /identity/v3/projects/83c74e06abe8467fae76793f536ae17a/users/fed6b7efe95141cf875944e41639f664/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:15.990069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:16.005404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15366919-670d-4bac-8407-26c53c9e9c6c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:16.056570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:16.269600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8ec0e-5aa1-4442-a644-abdefe08dcc5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:16.293153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:16.293719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f55e4-94d5-4f87-bbb8-4742c905db2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:16.294416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f55e4-94d5-4f87-bbb8-4742c905db2c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:16.299382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f55e4-94d5-4f87-bbb8-4742c905db2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:16.300044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:16.300402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f55e4-94d5-4f87-bbb8-4742c905db2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:16.300786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:16.301651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f55e4-94d5-4f87-bbb8-4742c905db2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:16.302056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:16.303063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:16.305487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:16.306819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f55e4-94d5-4f87-bbb8-4742c905db2c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:16.343110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 862/1716] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:16.351148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 855/1717] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:16.352947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:16.354002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:16.354417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:16.355366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:16.357913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:16.404037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5b8b5-acf8-4009-8432-34c373fb8246 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:16.416495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:16.417752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:16.427853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:16.428450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:16.428735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:16.429071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] PATH_INFO: `/v3/projects/b7f0e15e9b6b48229b9f6d3cc0ee107b` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:16.430785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f0e15e9b6b48229b9f6d3cc0ee107b)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:16.432825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:16.561523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f0e15e9b6b48229b9f6d3cc0ee107b', 'request_id': 'req-0ee3b104-e030-4439-93d1-119eccf786a1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:16.565758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 856/1718] 10.222.0.98 () {58 vars in 1015 bytes} [Mon Sep 26 17:59:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:16.566608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:16.574044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f0e15e9b6b48229b9f6d3cc0ee107b', 'request_id': 'req-0ee3b104-e030-4439-93d1-119eccf786a1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:16.585338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f0e15e9b6b48229b9f6d3cc0ee107b', 'request_id': 'req-0ee3b104-e030-4439-93d1-119eccf786a1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:16.615736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bd780-0366-4260-9940-9dad99f94738 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:16.620624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] The token cache is being invalidate because project b7f0e15e9b6b48229b9f6d3cc0ee107b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:16.621409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:16.624246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:16.641147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 863/1719] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:16 2022] DELETE /identity/v3/projects/b7f0e15e9b6b48229b9f6d3cc0ee107b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:16.642660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:16.651404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:16.673950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:16.892670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3b104-e030-4439-93d1-119eccf786a1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:16.900133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:16.900841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:16.913241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:16.913957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:16.914240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:16.914519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] PATH_INFO: `/v3/projects/83c74e06abe8467fae76793f536ae17a/users/fed6b7efe95141cf875944e41639f664/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:16.915993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:16.916237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] PATH_INFO: `/v3/projects/1827ccd6fa32432a8f4d3d4c7dec5139/users/c9b6ea720eb4420293d7b815927b0649/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:16.924923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c74e06abe8467fae76793f536ae17a, user_id=fed6b7efe95141cf875944e41639f664, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:16.941749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:16.986634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 864/1720] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:16 2022] PUT /identity/v3/projects/83c74e06abe8467fae76793f536ae17a/users/fed6b7efe95141cf875944e41639f664/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:17.001187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:17.002996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.016884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1827ccd6fa32432a8f4d3d4c7dec5139, user_id=c9b6ea720eb4420293d7b815927b0649, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:17.023654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:17.056498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:17.080432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 857/1721] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:16 2022] PUT /identity/v3/projects/1827ccd6fa32432a8f4d3d4c7dec5139/users/c9b6ea720eb4420293d7b815927b0649/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:17.082150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:17.098718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.147611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bf25f-6bb2-4962-99f2-5ec1d05cda81 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:17.349179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:17.349866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:17.350157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:17.350435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] PATH_INFO: `/v3/users/c1426375453f4b55bdc5bdd7501f9a08` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:17.361058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1426375453f4b55bdc5bdd7501f9a08)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:17.365550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:17.469396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555eb3d-0ea4-450e-80f5-4603b0a1ae66 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:17.474562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:17.475368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:17.477656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:17.478058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:17.483815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.515103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:17.516767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9be68-da66-43f9-82d8-807fd7fadca9 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:17.517881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1426375453f4b55bdc5bdd7501f9a08'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:17.518720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1426375453f4b55bdc5bdd7501f9a08'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:17.528646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 858/1722] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:17.530074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:17.530469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:17.530883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:17.537463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1426375453f4b55bdc5bdd7501f9a08'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:17.563725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1426375453f4b55bdc5bdd7501f9a08'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:17.565297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e65849b8-7206-4845-a850-3d39bb49ac63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1426375453f4b55bdc5bdd7501f9a08'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:17.589741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] MFA Rules not processed for user `fed6b7efe95141cf875944e41639f664`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:17.594383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.596031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 865/1723] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:16 2022] DELETE /identity/v3/users/c1426375453f4b55bdc5bdd7501f9a08 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:17.597470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:17.597761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:17.598042 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:17.663897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] MFA Rules not processed for user `c9b6ea720eb4420293d7b815927b0649`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:17.665480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.728541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 859/1724] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:17.729740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:17.731439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.771929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:17.851922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:17.874154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 866/1725] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:17.875547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:17.878005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:17.998220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12e72-9635-42c1-abb9-ef99d4caa093 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:18.012318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:18.012954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:18.013239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:18.013515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:18.022312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:18.062921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:18.068630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:18.097294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 860/1726] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:18.098039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:18.103174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:18.158209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77336d9e-00a5-4c46-9b35-5f405f859813 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:18.174131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:18.175629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:18.175629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:18.175629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:18.203718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:18.214251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:18.250146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 867/1727] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:18.326172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370d7e3-1a95-4d20-b89d-343e20771446 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:18.342109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7922e-5815-4074-aab1-644ff1a211ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:18.342814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7922e-5815-4074-aab1-644ff1a211ab service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:18.343108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7922e-5815-4074-aab1-644ff1a211ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:18.344472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7922e-5815-4074-aab1-644ff1a211ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:18.379858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7922e-5815-4074-aab1-644ff1a211ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:18.381352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7922e-5815-4074-aab1-644ff1a211ab service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:18.435004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 861/1728] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:19.984021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:19.986608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:20.174683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f1942-abef-4700-94a2-56839277a950 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:20.188746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:20.189733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:20.190366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:20.191080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] PATH_INFO: `/v3/users/8c0e8ff68e1847c99ba125c38ac0450e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:20.195870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0e8ff68e1847c99ba125c38ac0450e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:20.205277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:20.344361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0e8ff68e1847c99ba125c38ac0450e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:20.346777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0e8ff68e1847c99ba125c38ac0450e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:20.380595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0e8ff68e1847c99ba125c38ac0450e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:20.404688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0e8ff68e1847c99ba125c38ac0450e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:20.405917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0e8ff68e1847c99ba125c38ac0450e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:20.488007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 868/1729] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:19 2022] DELETE /identity/v3/users/8c0e8ff68e1847c99ba125c38ac0450e => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:20.492586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b85c-564d-470d-a0b6-4a8068dc94da None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:20.493813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b85c-564d-470d-a0b6-4a8068dc94da None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:20.494596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33b85c-564d-470d-a0b6-4a8068dc94da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:20.522339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2e33b85c-564d-470d-a0b6-4a8068dc94da None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:20.526719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2e33b85c-564d-470d-a0b6-4a8068dc94da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:20.776494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 862/1730] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:20.854604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:20.856740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:20.888007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:21.095514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4428da-8beb-4dfb-b6f8-1a3cc5397f17 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:21.136960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:21.138736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:21.139182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:21.142119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:21.167039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:21.184901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:21.210793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbfcf-7852-4233-9dd4-c960c72b388e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:21.211848 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbfcf-7852-4233-9dd4-c960c72b388e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:21.212055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dbfcf-7852-4233-9dd4-c960c72b388e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:21.262564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 869/1731] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:21.267475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f79dbfcf-7852-4233-9dd4-c960c72b388e None None] MFA Rules not processed for user `4bbd5a38a6ce472790128a37f62d472b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:21.268871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f79dbfcf-7852-4233-9dd4-c960c72b388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:21.462241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 863/1732] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:21.750981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:21.754030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:21.778443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c2690-ed32-40e7-87cf-0790030ed237 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:21.793277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:21.793938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:21.794225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:21.794502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] PATH_INFO: `/v3/projects/cc4114a9ef274f51a518428a68fb64cb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:21.795745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc4114a9ef274f51a518428a68fb64cb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:21.802823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:21.933515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc4114a9ef274f51a518428a68fb64cb', 'request_id': 'req-7089078d-b8d4-4481-82f7-e0f8a318f2f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:21.943772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc4114a9ef274f51a518428a68fb64cb', 'request_id': 'req-7089078d-b8d4-4481-82f7-e0f8a318f2f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:21.961509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc4114a9ef274f51a518428a68fb64cb', 'request_id': 'req-7089078d-b8d4-4481-82f7-e0f8a318f2f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:21.993319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] The token cache is being invalidate because project cc4114a9ef274f51a518428a68fb64cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:21.993632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:21.999271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7089078d-b8d4-4481-82f7-e0f8a318f2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:22.001987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 870/1733] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:21 2022] DELETE /identity/v3/projects/cc4114a9ef274f51a518428a68fb64cb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:22.009480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:22.010173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:22.010690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:22.088222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:22.092842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:22.159628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:22.408231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 864/1734] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:22.416763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:22.417212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:22.417936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:22.480355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:22.484197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:22.554274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 871/1735] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:22.555486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:22.570786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd7312-8abf-4c6c-afca-0d5d9343cf95 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:22.589307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:22.592625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:22.592914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:22.593208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:22.594510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:22.608971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:22.616213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:22.665455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:22.673495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 865/1736] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:22.677990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db4563-131e-480a-aeb1-9495460dd301 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:22.704799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:22.708412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:22.711000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:22.712559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:22.714097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:22.721806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:22.806087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 872/1737] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:22 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:22.811222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:22.826975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496c728-1412-4051-864c-2859a3b79708 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.040255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.041368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.042466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.044289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.045250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.053888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.108468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 866/1738] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:22 2022] POST /identity/v3/users => generated 288 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:23.108671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.125564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a9705-6165-428f-a9eb-fcbb3dd5507c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.144695 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.145573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.146022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.147117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.148135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.149897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.198430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 873/1739] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:23.202128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.216603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863f2a1-44f1-4597-8702-c220d1e8f568 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.225170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f5cfc-5063-4135-a513-ce5dd357c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.226250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5cfc-5063-4135-a513-ce5dd357c9db admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.226745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5cfc-5063-4135-a513-ce5dd357c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.227287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f5cfc-5063-4135-a513-ce5dd357c9db admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.228158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f5cfc-5063-4135-a513-ce5dd357c9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.230218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f5cfc-5063-4135-a513-ce5dd357c9db admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.246912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 867/1740] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:23.251135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.272613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e7d81-fcb4-4707-86b7-86bb59586a02 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.279350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.280404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.281445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.282102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] PATH_INFO: `/v3/projects/341ec0fef01c4c42ad0934ffe973fb8a/users/5021cbef30f84af1a1cfcfde16aeeb54/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.316926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=341ec0fef01c4c42ad0934ffe973fb8a, user_id=5021cbef30f84af1a1cfcfde16aeeb54, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.325766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.365508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 874/1741] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:23 2022] PUT /identity/v3/projects/341ec0fef01c4c42ad0934ffe973fb8a/users/5021cbef30f84af1a1cfcfde16aeeb54/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:23.375739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.376385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.376842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.423901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] MFA Rules not processed for user `5021cbef30f84af1a1cfcfde16aeeb54`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:23.425594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:23.534620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 868/1742] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:23.539970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.618057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:23.699750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3696ac-c129-4646-8d59-ee2b0b36b3da admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.706362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.707760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.708259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.708739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.709597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.712217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.779287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 875/1743] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:23 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:23.780667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.803545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19dbc92-728b-43bd-92e3-49a396c1c3a5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.840705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:23.987965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.988803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.989181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.989549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.990603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.993648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.061381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 869/1744] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:23 2022] POST /identity/v3/users => generated 295 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:24.066127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.084776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376bb2-6cdd-456f-8110-1e8ef3109377 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.096298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.099225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.099620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.100003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.100792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.103117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.128557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 876/1745] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:24.132441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.152750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceaa25-0c9d-4a47-9b9b-7b6365a2d45a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.159531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.160222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.160555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.160827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] PATH_INFO: `/v3/projects/4447b9c7cc454fd7b29324a5038cd98e/users/5ab1de7bf92e4aa3988050a2e8a3f77a/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.189769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4447b9c7cc454fd7b29324a5038cd98e, user_id=5ab1de7bf92e4aa3988050a2e8a3f77a, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.192801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.236850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.245183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 870/1746] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:24 2022] PUT /identity/v3/projects/4447b9c7cc454fd7b29324a5038cd98e/users/5ab1de7bf92e4aa3988050a2e8a3f77a/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:24.254256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab07b9-f614-4735-ac69-49d9ef131bea admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.419475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9927f64-2e06-4f9d-b9db-b41ebffe94e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.420106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9927f64-2e06-4f9d-b9db-b41ebffe94e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.420388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9927f64-2e06-4f9d-b9db-b41ebffe94e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.420664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9927f64-2e06-4f9d-b9db-b41ebffe94e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.434152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9927f64-2e06-4f9d-b9db-b41ebffe94e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.434152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9927f64-2e06-4f9d-b9db-b41ebffe94e4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.445837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 877/1747] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:24.452479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.486276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492aa47d-8f89-48e1-a462-69569b30f1df admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.493794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d1d0a-5d00-4051-b519-b2e3c3545156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.494657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d1d0a-5d00-4051-b519-b2e3c3545156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.494995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d1d0a-5d00-4051-b519-b2e3c3545156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.495497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d1d0a-5d00-4051-b519-b2e3c3545156 admin admin] PATH_INFO: `/v3/projects/4447b9c7cc454fd7b29324a5038cd98e/users/5ab1de7bf92e4aa3988050a2e8a3f77a/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.505427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d1d0a-5d00-4051-b519-b2e3c3545156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4447b9c7cc454fd7b29324a5038cd98e, user_id=5ab1de7bf92e4aa3988050a2e8a3f77a, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.508741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d1d0a-5d00-4051-b519-b2e3c3545156 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.539187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 871/1748] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:24 2022] PUT /identity/v3/projects/4447b9c7cc454fd7b29324a5038cd98e/users/5ab1de7bf92e4aa3988050a2e8a3f77a/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:24.539828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dda25-d0c1-4758-a5c8-10b146c24185 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.540221 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dda25-d0c1-4758-a5c8-10b146c24185 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.540872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dda25-d0c1-4758-a5c8-10b146c24185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.625401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-891dda25-d0c1-4758-a5c8-10b146c24185 None None] MFA Rules not processed for user `5ab1de7bf92e4aa3988050a2e8a3f77a`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:24.626680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-891dda25-d0c1-4758-a5c8-10b146c24185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:24.756956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d34f9-7100-4c1f-ad83-e6aa9138fc0b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.757247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d34f9-7100-4c1f-ad83-e6aa9138fc0b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.757521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d34f9-7100-4c1f-ad83-e6aa9138fc0b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.759958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 872/1749] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 17:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:24.787108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 878/1750] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:24.798788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d34f9-7100-4c1f-ad83-e6aa9138fc0b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.800611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba9d34f9-7100-4c1f-ad83-e6aa9138fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.010988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d34f9-7100-4c1f-ad83-e6aa9138fc0b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:25.030304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2bb1b-8f97-418a-8d26-984e0e8d8676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:25.034952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2bb1b-8f97-418a-8d26-984e0e8d8676 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.035255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2bb1b-8f97-418a-8d26-984e0e8d8676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.035533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2bb1b-8f97-418a-8d26-984e0e8d8676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.108493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69361-9733-410d-9ce4-41f6627951ed None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.108951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69361-9733-410d-9ce4-41f6627951ed None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.110000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed69361-9733-410d-9ce4-41f6627951ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.176072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6ed69361-9733-410d-9ce4-41f6627951ed None None] MFA Rules not processed for user `28195a63990344edb6ea31f1f4463a82`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:25.177825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6ed69361-9733-410d-9ce4-41f6627951ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.243101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:25.277271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2bb1b-8f97-418a-8d26-984e0e8d8676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:25.278751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2bb1b-8f97-418a-8d26-984e0e8d8676 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:25.312051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 873/1751] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:25.314773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.315448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.315733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.364447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:25.365832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.385139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 879/1752] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:25.386712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5ee7b-09ea-43ba-ab68-22ab2caf67aa None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.387238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5ee7b-09ea-43ba-ab68-22ab2caf67aa None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.387735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5ee7b-09ea-43ba-ab68-22ab2caf67aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.410992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:25.455914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-44a5ee7b-09ea-43ba-ab68-22ab2caf67aa None None] MFA Rules not processed for user `8d562e923af0446db67404ce4ad36cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:25.457533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-44a5ee7b-09ea-43ba-ab68-22ab2caf67aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.560315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 874/1753] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 17:59:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:25.561454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:25.565579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.658593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 880/1754] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:25.669679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.670117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.670502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.756697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] MFA Rules not processed for user `8d562e923af0446db67404ce4ad36cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:25.763915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.773089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b41f8-e3f8-4983-a5a2-91a88238dd2f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:25.788364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:25.789366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.790086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.790554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] PATH_INFO: `/v3/projects/44ec15b901f942cb9a9cc57541cfb201` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.795967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 881/1755] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:25.802806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ec15b901f942cb9a9cc57541cfb201)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:25.808322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:25.965176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ec15b901f942cb9a9cc57541cfb201', 'request_id': 'req-17025bed-984f-4418-b548-214ac53c5226'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:25.982800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ec15b901f942cb9a9cc57541cfb201', 'request_id': 'req-17025bed-984f-4418-b548-214ac53c5226'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.004020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ec15b901f942cb9a9cc57541cfb201', 'request_id': 'req-17025bed-984f-4418-b548-214ac53c5226'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.041076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] The token cache is being invalidate because project 44ec15b901f942cb9a9cc57541cfb201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:26.041395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.042527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-17025bed-984f-4418-b548-214ac53c5226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.044813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 875/1756] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:25 2022] DELETE /identity/v3/projects/44ec15b901f942cb9a9cc57541cfb201 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:26.048987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:26.069201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a02e9-15c5-47a1-bbb5-b7feccd093d8 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:26.148661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1257-9c19-4228-83f2-f0e6dd4bd563 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.149010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1257-9c19-4228-83f2-f0e6dd4bd563 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.149314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1257-9c19-4228-83f2-f0e6dd4bd563 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:26.151172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 876/1757] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 17:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:26.185267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:26.186913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.187964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.188399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] PATH_INFO: `/v3/users/fed6b7efe95141cf875944e41639f664` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:26.190640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed6b7efe95141cf875944e41639f664)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:26.193045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:26.291722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed6b7efe95141cf875944e41639f664'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.293614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed6b7efe95141cf875944e41639f664'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.310321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:26.318672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed6b7efe95141cf875944e41639f664'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.346753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed6b7efe95141cf875944e41639f664'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.347541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed6b7efe95141cf875944e41639f664'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:26.379585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 882/1758] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:26 2022] DELETE /identity/v3/users/fed6b7efe95141cf875944e41639f664 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:26.382232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.382728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.382809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:26.448514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:26.452316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:26.526147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:26.527636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:26.563366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:26.632576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 877/1759] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 17:59:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:26.714075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1377d7b-f4b3-4dd6-a9ad-56c986869be0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:26.725657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:26.726592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.727028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.727395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:26.730383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:26.753340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:26.756000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e4461-950d-467d-9acc-dda34cfbaf2e service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:26.774658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 883/1760] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:26.896718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:26.924040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9604ba-83d6-49d3-be06-b4e8aa03bc5f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:26.941461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:26.942080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.942626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.942958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:26.944979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:26.975698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc4250-c82b-4f9c-b6b1-63dd72866851 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.976688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc4250-c82b-4f9c-b6b1-63dd72866851 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.983721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc4250-c82b-4f9c-b6b1-63dd72866851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:27.049298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-76fc4250-c82b-4f9c-b6b1-63dd72866851 None None] MFA Rules not processed for user `276174ef02b1409b840a0de13aa4095d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:27.050582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-76fc4250-c82b-4f9c-b6b1-63dd72866851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:27.152650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:27.158010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd128da3-70f6-4d65-9e8c-15443d4c4fa4 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:27.172703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 884/1761] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:27.204559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 878/1762] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:27.827685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf04ea-f5d0-47bf-96e7-ef777433de47 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:27.828982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf04ea-f5d0-47bf-96e7-ef777433de47 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:27.829490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf04ea-f5d0-47bf-96e7-ef777433de47 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:27.832681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 885/1763] 10.222.0.72 () {54 vars in 912 bytes} [Mon Sep 26 17:59:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:27.839239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:27.840415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:27.840920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:27.874976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:27.876591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:27.940284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 879/1764] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 17:59:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:27.965191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf04ea-f5d0-47bf-96e7-ef777433de47 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:27.969386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5aaf04ea-f5d0-47bf-96e7-ef777433de47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:27.997372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf04ea-f5d0-47bf-96e7-ef777433de47 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.002738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df560f8b-aaec-432e-83b7-65d340d07d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.003645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df560f8b-aaec-432e-83b7-65d340d07d24 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.004328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df560f8b-aaec-432e-83b7-65d340d07d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.004709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df560f8b-aaec-432e-83b7-65d340d07d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.013705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df560f8b-aaec-432e-83b7-65d340d07d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.015900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df560f8b-aaec-432e-83b7-65d340d07d24 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.026026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 886/1765] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:28.048756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.050301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:28.127397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcdf6b-6e30-4aa2-b1b0-9fcfa8417d17 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.133067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.133958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.134244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.134670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] PATH_INFO: `/v3/projects/83c74e06abe8467fae76793f536ae17a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.135982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c74e06abe8467fae76793f536ae17a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.138047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.178063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:28.216732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c74e06abe8467fae76793f536ae17a', 'request_id': 'req-67e85a26-8928-41c0-a629-b14e2a56e078'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:28.225490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c74e06abe8467fae76793f536ae17a', 'request_id': 'req-67e85a26-8928-41c0-a629-b14e2a56e078'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:28.234967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c74e06abe8467fae76793f536ae17a', 'request_id': 'req-67e85a26-8928-41c0-a629-b14e2a56e078'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:28.249024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] The token cache is being invalidate because project 83c74e06abe8467fae76793f536ae17a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:28.249981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:28.250091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e85a26-8928-41c0-a629-b14e2a56e078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:28.252169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 880/1766] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:28 2022] DELETE /identity/v3/projects/83c74e06abe8467fae76793f536ae17a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:28.256912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.257056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.257336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.281841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:28.283134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:28.341256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:28.369819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 887/1767] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:28.372989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.373265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.373548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.396137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:28.398695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:28.416031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 881/1768] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:28.416789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.430151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99813cbb-72fa-4714-9cd0-661ef798b36a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.436407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.437644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.437812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.438090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.439401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:28.439675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.441862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.463317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 888/1769] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:28.464762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.475927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9c7e9-9f4b-4844-be55-b8744c00d429 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.480922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.481561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.481908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.482251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.482996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.486318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.533996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 882/1770] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:28 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:28.535351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.548607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fab33-1f5e-495a-af08-05028aba4633 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.672690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.673894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.674553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.675715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.676577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.679121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.721736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 889/1771] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:28 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:28.725583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.742313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368c121-b861-440d-a5f3-f1e11ab6f923 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.759517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.760354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.760840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.761301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.762339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.764841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.775333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.778271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:28.788199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 883/1772] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:28.789141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.813246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cef69-160a-43c2-9e52-a4dbd7097b7e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.837716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.838355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.839092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.839370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.840023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.850017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.881056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 884/1773] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:28.885611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:28.916344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eb314-5a0e-4893-a808-c6b5fda3f732 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:28.929974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41323e67-f5d3-472a-8d02-aebc28c8374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:28.930929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323e67-f5d3-472a-8d02-aebc28c8374f admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:28.931328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323e67-f5d3-472a-8d02-aebc28c8374f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:28.932730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323e67-f5d3-472a-8d02-aebc28c8374f admin admin] PATH_INFO: `/v3/projects/2487b6670a584c2281733fa2b490e135/users/178000341af94bbfaf1c36f3583471db/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:28.955757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323e67-f5d3-472a-8d02-aebc28c8374f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2487b6670a584c2281733fa2b490e135, user_id=178000341af94bbfaf1c36f3583471db, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:28.960981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323e67-f5d3-472a-8d02-aebc28c8374f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:28.989731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 885/1774] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:28 2022] PUT /identity/v3/projects/2487b6670a584c2281733fa2b490e135/users/178000341af94bbfaf1c36f3583471db/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:59:28.998867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8029a13-eefa-4e12-874f-971e70d0f50d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:29.005362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:29.005659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:29.008221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:29.064953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] MFA Rules not processed for user `178000341af94bbfaf1c36f3583471db`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:29.066644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:29.205638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:29.227981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:29.228870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:29.229224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:29.231403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:29.233890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:29.311894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:29.314588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:29.318045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 886/1775] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:29.333959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:29.337273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:29.357483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 890/1776] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:29.423946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:29.473832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade81a-df54-4cb7-93a3-a153431a4821 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:29.490233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323fb02-c248-4a0c-bf3c-00fb6f910497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:29.490935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323fb02-c248-4a0c-bf3c-00fb6f910497 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:29.491694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323fb02-c248-4a0c-bf3c-00fb6f910497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:29.491962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323fb02-c248-4a0c-bf3c-00fb6f910497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:29.509702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323fb02-c248-4a0c-bf3c-00fb6f910497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:29.513733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0323fb02-c248-4a0c-bf3c-00fb6f910497 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:29.543246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 887/1777] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:29.567979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:29.693037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffbc22-1559-4391-9fc3-dfaf82d6a874 service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:29.722121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:29.722816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:29.723214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:29.724291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] PATH_INFO: `/v3/users/178000341af94bbfaf1c36f3583471db` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:29.727594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178000341af94bbfaf1c36f3583471db)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:29.729735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:29.812022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178000341af94bbfaf1c36f3583471db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:29.813062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178000341af94bbfaf1c36f3583471db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:29.822697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178000341af94bbfaf1c36f3583471db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:29.839728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178000341af94bbfaf1c36f3583471db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:29.841267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178000341af94bbfaf1c36f3583471db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:29.881569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 891/1778] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:29 2022] DELETE /identity/v3/users/178000341af94bbfaf1c36f3583471db => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:29.883361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:29.883801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:29.884740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:29.912184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:29.913923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:30.004604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 888/1779] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:30.012056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:30.014607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:30.115137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4797fba-0ea6-4242-b8b1-260101bc25e8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:30.119843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89e9cd-b01a-42d5-9697-ec5b5af65579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:30.120553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89e9cd-b01a-42d5-9697-ec5b5af65579 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:30.120906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89e9cd-b01a-42d5-9697-ec5b5af65579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:30.121253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89e9cd-b01a-42d5-9697-ec5b5af65579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:30.137902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89e9cd-b01a-42d5-9697-ec5b5af65579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:30.139424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89e9cd-b01a-42d5-9697-ec5b5af65579 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:30.173541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 892/1780] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:30.764978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:30.767416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:30.787630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6235833-0902-482b-b045-6c574a2af9cd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:30.793254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:30.794201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:30.794514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:30.794802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] PATH_INFO: `/v3/projects/2487b6670a584c2281733fa2b490e135` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:30.796026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2487b6670a584c2281733fa2b490e135)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:30.808533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:30.894422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2487b6670a584c2281733fa2b490e135', 'request_id': 'req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:30.902456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2487b6670a584c2281733fa2b490e135', 'request_id': 'req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:30.915638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2487b6670a584c2281733fa2b490e135', 'request_id': 'req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:30.928184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] The token cache is being invalidate because project 2487b6670a584c2281733fa2b490e135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:30.928347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:30.929101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7614b6d0-a61d-4e21-afee-fd21b1cdb59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:30.934463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 889/1781] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:30 2022] DELETE /identity/v3/projects/2487b6670a584c2281733fa2b490e135 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:30.936727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:30.937853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:30.938929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:30.969348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:30.970645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:30.978436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:31.058989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 893/1782] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:31.065206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.065567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.065912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.089403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:31.090624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:31.099553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 890/1783] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:31.103783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.114023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f82b-eb47-4481-86e7-611af693d5d3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.118793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.119546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.119909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.120247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.120989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:31.121350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.123151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.138720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 894/1784] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:31.139983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.152936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a712468-3fb3-4b6f-a09f-0ef8cfb8dd5d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.157758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.158366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.158646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.158955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.159589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.161934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.194336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 891/1785] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:31 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:31.195594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.206591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24348a-8881-4734-aeb7-35d9da74ef98 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.274725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.275383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.275667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.275944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.276818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.278761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.307402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 895/1786] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:31 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:31.308620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.321618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defd4fe-ac57-45f9-bb6b-29afec875bcf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.327781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.328548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.328928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.329087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.329732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.332153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.342504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 892/1787] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:31.346771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.356647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbce6c3-825c-4eab-ab38-7765afa5964b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.361884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebb5b-9a8c-47a7-8109-44fce94ff109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.362472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebb5b-9a8c-47a7-8109-44fce94ff109 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.362755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebb5b-9a8c-47a7-8109-44fce94ff109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.363061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebb5b-9a8c-47a7-8109-44fce94ff109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.363698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efebb5b-9a8c-47a7-8109-44fce94ff109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.365273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efebb5b-9a8c-47a7-8109-44fce94ff109 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.376635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 896/1788] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 17:59:31.380817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.390788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ee1dc-1e4f-4082-baae-45a6eb137412 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.395833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.396419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.396701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.396977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] PATH_INFO: `/v3/projects/c67eedd83a584d63815004b50d67f44a/users/3c60ede95d154bc199f2f6495b3789f9/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.408115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:31.413545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c67eedd83a584d63815004b50d67f44a, user_id=3c60ede95d154bc199f2f6495b3789f9, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.417693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.431179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 893/1789] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:31 2022] PUT /identity/v3/projects/c67eedd83a584d63815004b50d67f44a/users/3c60ede95d154bc199f2f6495b3789f9/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:31.442701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.443015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.443316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.471316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] MFA Rules not processed for user `3c60ede95d154bc199f2f6495b3789f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:31.472522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:31.536176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 897/1790] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:31.548813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.550453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:31.616195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1a6a0-d38d-4e56-b592-82200b0394cc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.622887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.623528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.623816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.624949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.637047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:31.639427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:31.652440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 894/1791] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:31.860791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:31.862990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:31.928656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0196f-89c2-4180-b660-04f0279b5773 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:31.942009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:31.943008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:31.943362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:31.943792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:31.946372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:31.967128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:32.098076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:32.100305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd909f49-0c31-4681-bd8f-cd23707b6abe service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:32.122141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 898/1792] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:59:40.355851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:40.357587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:40.518763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714ea19-14a6-4253-8db0-bf699c6f6534 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:40.524944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:40.525593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:40.525959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:40.526428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] PATH_INFO: `/v3/users/c9b6ea720eb4420293d7b815927b0649` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:40.527985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b6ea720eb4420293d7b815927b0649)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:40.530154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:40.605888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b6ea720eb4420293d7b815927b0649'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:40.611288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b6ea720eb4420293d7b815927b0649'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:40.611949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b6ea720eb4420293d7b815927b0649'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:40.648532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b6ea720eb4420293d7b815927b0649'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:40.673576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b6ea720eb4420293d7b815927b0649'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:40.690575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 895/1793] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:40 2022] DELETE /identity/v3/users/c9b6ea720eb4420293d7b815927b0649 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:40.696446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:40.697324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:40.698825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:40.724434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:40.725883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:40.839396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 899/1794] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:40.850673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:40.852245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:40.871292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:40.925819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb504b4-aedf-423a-9777-25ebca85f347 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:40.929450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22204b-e988-4ee9-8674-c0592df12676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:40.930062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22204b-e988-4ee9-8674-c0592df12676 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:40.930344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22204b-e988-4ee9-8674-c0592df12676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:40.930612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22204b-e988-4ee9-8674-c0592df12676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:40.939756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22204b-e988-4ee9-8674-c0592df12676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:40.941189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22204b-e988-4ee9-8674-c0592df12676 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:40.951116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 896/1795] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:41.245621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.245621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:41.261353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f4e00-b327-4b6f-8105-85aed51ccb19 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.267754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.269067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.270133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.270133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] PATH_INFO: `/v3/projects/1827ccd6fa32432a8f4d3d4c7dec5139` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.273352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1827ccd6fa32432a8f4d3d4c7dec5139)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.290407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.373531 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1827ccd6fa32432a8f4d3d4c7dec5139', 'request_id': 'req-88819eab-1cb3-4850-be0b-529651e9ca27'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:41.386803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1827ccd6fa32432a8f4d3d4c7dec5139', 'request_id': 'req-88819eab-1cb3-4850-be0b-529651e9ca27'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:41.401501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1827ccd6fa32432a8f4d3d4c7dec5139', 'request_id': 'req-88819eab-1cb3-4850-be0b-529651e9ca27'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:41.425933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] The token cache is being invalidate because project 1827ccd6fa32432a8f4d3d4c7dec5139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:41.426128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:41.426786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88819eab-1cb3-4850-be0b-529651e9ca27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:41.428313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 900/1796] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:41 2022] DELETE /identity/v3/projects/1827ccd6fa32432a8f4d3d4c7dec5139 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:41.433318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.434159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.434582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.456312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:41.457809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:41.529884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 897/1797] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:41.538415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.538729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.539029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.560198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:41.561680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:41.571196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 901/1798] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:41.575447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.586054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23233e1-3cc6-4816-a3ae-5e11f8a04bbb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.590977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.592636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.592915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.593189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.595026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:41.595291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.597125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.611201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 898/1799] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:41.615666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.625150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd950a-4927-40db-b68f-aa4dc6fd1776 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.630920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.631526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.631809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.632089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.632730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.634644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.650897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:41.666860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 902/1800] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:41 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:41.670854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.686308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad2250-2fc0-4c90-baf1-8470ab7c667e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.744815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.745424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.745701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.745973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.746601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.748577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.770431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 899/1801] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:41 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:41.774916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.786150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be846081-cfab-4ad3-9b90-f107af28aa87 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.791762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.792391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.792674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.792951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.793588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.795256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.805110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 903/1802] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:41.809149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.819421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651b371-b59a-4d6d-8484-4fe6157e6db2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.824167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b382bc3-bdd5-4b3a-ab13-3382509ffaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.824764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382bc3-bdd5-4b3a-ab13-3382509ffaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.825042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382bc3-bdd5-4b3a-ab13-3382509ffaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.825338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b382bc3-bdd5-4b3a-ab13-3382509ffaf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.825972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b382bc3-bdd5-4b3a-ab13-3382509ffaf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.828413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b382bc3-bdd5-4b3a-ab13-3382509ffaf2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.839985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 900/1803] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:41.844617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:41.854614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eea612-6572-43de-ad1a-ea5ff369ede1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:41.859350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:41.860101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.860414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.860714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] PATH_INFO: `/v3/projects/0ac80b86ab4f4d14bdc0c53e079b5213/users/b6a6e9ed617042d2bf4c059356e36325/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.875626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac80b86ab4f4d14bdc0c53e079b5213, user_id=b6a6e9ed617042d2bf4c059356e36325, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:41.878307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:41.892306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 904/1804] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:41 2022] PUT /identity/v3/projects/0ac80b86ab4f4d14bdc0c53e079b5213/users/b6a6e9ed617042d2bf4c059356e36325/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:41.902629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:41.902950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:41.903228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:41.935113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] MFA Rules not processed for user `b6a6e9ed617042d2bf4c059356e36325`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:41.936538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:41.945867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:42.008730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 901/1805] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:42.016913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.079520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e0bd-0629-406d-be0e-f311b247265c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.084834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.085431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.085718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.085994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.086628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.088583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.115180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 905/1806] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:42 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:42.119513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.131084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650e8e-eb8d-4977-8550-4afc3a8a0874 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.195425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.196034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.196313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.196587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.197205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.199405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.221799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 902/1807] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:42 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:42.225872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.236228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c2e89-bf13-42a4-8e17-cfb4c08b49a0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.240824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.241420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.241703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.241977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.242611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.244245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.255356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 906/1808] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:42.259457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.269995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29939f-4728-49d9-b89f-86ea2cf9e360 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.274627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.275312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.275593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.275868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] PATH_INFO: `/v3/projects/a4a2ca7daf6b44d9a76b6c4f6ca9658b/users/d0021a9073d6489f961bdad5cc053e6d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.290319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a2ca7daf6b44d9a76b6c4f6ca9658b, user_id=d0021a9073d6489f961bdad5cc053e6d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.292999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.305891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 903/1809] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:42 2022] PUT /identity/v3/projects/a4a2ca7daf6b44d9a76b6c4f6ca9658b/users/d0021a9073d6489f961bdad5cc053e6d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:42.309868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.321051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb88d14-1560-456e-8138-0022ad1a186b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.380906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd2ddb-946c-46c8-aa73-e77afa1e4333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.381431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd2ddb-946c-46c8-aa73-e77afa1e4333 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.381719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd2ddb-946c-46c8-aa73-e77afa1e4333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.381998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd2ddb-946c-46c8-aa73-e77afa1e4333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.382673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd2ddb-946c-46c8-aa73-e77afa1e4333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.384306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd2ddb-946c-46c8-aa73-e77afa1e4333 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.398140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 907/1810] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:42.399316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.411250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ab262-5a2c-4d53-835a-b0e6d919cf28 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.415891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.416551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.416833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.417103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] PATH_INFO: `/v3/projects/a4a2ca7daf6b44d9a76b6c4f6ca9658b/users/d0021a9073d6489f961bdad5cc053e6d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.419051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a2ca7daf6b44d9a76b6c4f6ca9658b, user_id=d0021a9073d6489f961bdad5cc053e6d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.422230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.433218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 904/1811] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:42 2022] PUT /identity/v3/projects/a4a2ca7daf6b44d9a76b6c4f6ca9658b/users/d0021a9073d6489f961bdad5cc053e6d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:59:42.445549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.445922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.446278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.474722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] MFA Rules not processed for user `d0021a9073d6489f961bdad5cc053e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:42.476101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:42.496768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:42.559251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 908/1812] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:42.571388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.573368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:42.634853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882da2-889c-4695-8f49-94544a510548 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.638428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c385dbf-5cb3-4afb-bc6d-fafdb168a7b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.639233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c385dbf-5cb3-4afb-bc6d-fafdb168a7b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.639521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c385dbf-5cb3-4afb-bc6d-fafdb168a7b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.639798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c385dbf-5cb3-4afb-bc6d-fafdb168a7b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.649495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c385dbf-5cb3-4afb-bc6d-fafdb168a7b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.650932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c385dbf-5cb3-4afb-bc6d-fafdb168a7b0 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.661552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 905/1813] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:42.746604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:42.748073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:42.760280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae090618-1e14-4ece-8da3-480d14cef75c None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:42.764158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db385e7-6204-42b4-a9d6-0fd8a73ad616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:42.764769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db385e7-6204-42b4-a9d6-0fd8a73ad616 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:42.765118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db385e7-6204-42b4-a9d6-0fd8a73ad616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:42.765360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db385e7-6204-42b4-a9d6-0fd8a73ad616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:42.822463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db385e7-6204-42b4-a9d6-0fd8a73ad616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:42.824167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db385e7-6204-42b4-a9d6-0fd8a73ad616 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:42.832948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 909/1814] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:44.558829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb567-a89f-42d2-b2cb-76dfa10e4958 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:44.559380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb567-a89f-42d2-b2cb-76dfa10e4958 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:44.560114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb567-a89f-42d2-b2cb-76dfa10e4958 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:44.562954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 906/1815] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 17:59:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:44.574740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:44.576101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:44.576620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:44.617175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:44.621873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:44.791124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 910/1816] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 17:59:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:44.969224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffb567-a89f-42d2-b2cb-76dfa10e4958 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:44.971662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-edffb567-a89f-42d2-b2cb-76dfa10e4958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:44.997997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:45.014922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffb567-a89f-42d2-b2cb-76dfa10e4958 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:45.032863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aebec-db6b-4445-92ef-914ef913123f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:45.033804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aebec-db6b-4445-92ef-914ef913123f service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:45.034990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aebec-db6b-4445-92ef-914ef913123f service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:45.035529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aebec-db6b-4445-92ef-914ef913123f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:45.059676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aebec-db6b-4445-92ef-914ef913123f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:45.065222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aebec-db6b-4445-92ef-914ef913123f service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:45.088712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 907/1817] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:46.173741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:46.176133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:46.293628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:46.326711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826abebf-1c35-41a1-ac76-e75e8081012f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:46.333549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:46.335917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:46.336526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:46.337117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] PATH_INFO: `/v3/users/3c60ede95d154bc199f2f6495b3789f9` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:46.338685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c60ede95d154bc199f2f6495b3789f9)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:46.349374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:46.417064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c60ede95d154bc199f2f6495b3789f9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:46.418147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c60ede95d154bc199f2f6495b3789f9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:46.429985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c60ede95d154bc199f2f6495b3789f9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:46.449343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c60ede95d154bc199f2f6495b3789f9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:46.450027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c60ede95d154bc199f2f6495b3789f9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:46.476238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 911/1818] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:46 2022] DELETE /identity/v3/users/3c60ede95d154bc199f2f6495b3789f9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:46.476533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:46.477211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:46.477456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:46.514909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:46.517585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:46.678978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 908/1819] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:46.691093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:46.692956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:46.776778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2982a8-c1da-4914-ae4c-f0f0f4cb50a6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:46.782235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113b4ef-4de4-410e-9a2b-cf0c0495dd52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:46.783011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113b4ef-4de4-410e-9a2b-cf0c0495dd52 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:46.783298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113b4ef-4de4-410e-9a2b-cf0c0495dd52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:46.783578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113b4ef-4de4-410e-9a2b-cf0c0495dd52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:46.811500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113b4ef-4de4-410e-9a2b-cf0c0495dd52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:46.814717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113b4ef-4de4-410e-9a2b-cf0c0495dd52 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:46.829106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 912/1820] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:47.326444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:47.329195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:47.360060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccf30f-bb24-4761-a848-6c85f892d721 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:47.366028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:47.367705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.368489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.368782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] PATH_INFO: `/v3/projects/c67eedd83a584d63815004b50d67f44a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.370241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c67eedd83a584d63815004b50d67f44a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:47.372235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:47.434299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c67eedd83a584d63815004b50d67f44a', 'request_id': 'req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:47.442492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c67eedd83a584d63815004b50d67f44a', 'request_id': 'req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:47.452191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c67eedd83a584d63815004b50d67f44a', 'request_id': 'req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:47.470690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] The token cache is being invalidate because project c67eedd83a584d63815004b50d67f44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:47.471013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:47.471796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0523a6a0-a659-48dd-9749-a1a1fc9eac6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:47.473359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 909/1821] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:47 2022] DELETE /identity/v3/projects/c67eedd83a584d63815004b50d67f44a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:47.478597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.479088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.479372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.502190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:47.503746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:47.587149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 913/1822] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:47.591552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.592109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.592505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.620784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:47.622274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:47.658419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 910/1823] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:47.658559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:47.678355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8792d53-4fec-4f36-990c-405d08c372b5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:47.690306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:47.690966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.691623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.692202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.693083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:47.693607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:47.695371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:47.699341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:47.721305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:47.728044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 914/1824] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:59:47.736947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4db9d-19a5-47e0-adcf-87c9a46e0bf8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:47.742458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:47.743688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.744146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.744604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.745554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:47.747703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:47.783788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 911/1825] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:47 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:47.788325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:47.800396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489702d-d67c-4b51-afd1-91b0ff84cfe3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:47.894043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:47.894665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.895503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.895503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.895900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:47.897825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:47.927032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 915/1826] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:47 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:47.928606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:47.940037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f938c3-04bd-48f3-a32a-56537aa02ccd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:47.958582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:47.959331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.959676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.960016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:47.960661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:47.962241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:47.974431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 912/1827] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 17:59:47.979016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:47.993092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398aa436-dab9-4dbc-9d19-d190b35ac6d1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:47.998214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d18ef-b6fb-4540-9f1f-b8b6944562fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:47.998946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d18ef-b6fb-4540-9f1f-b8b6944562fe admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:47.999231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d18ef-b6fb-4540-9f1f-b8b6944562fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:47.999535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d18ef-b6fb-4540-9f1f-b8b6944562fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:48.000287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d18ef-b6fb-4540-9f1f-b8b6944562fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:48.001795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d18ef-b6fb-4540-9f1f-b8b6944562fe admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:48.013205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 916/1828] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:48.017537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:48.028140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90c2da-c4bc-4752-b3e1-d573a93621d6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:48.033840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:48.034499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:48.034776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:48.035078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] PATH_INFO: `/v3/projects/f50038fc43984b8e902487ac2e58e5e8/users/300a2341d082407a97458f4a144161af/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:48.053985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50038fc43984b8e902487ac2e58e5e8, user_id=300a2341d082407a97458f4a144161af, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:48.056726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:48.070505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 913/1829] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:48 2022] PUT /identity/v3/projects/f50038fc43984b8e902487ac2e58e5e8/users/300a2341d082407a97458f4a144161af/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:59:48.082630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:48.083018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:48.083367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:48.113613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] MFA Rules not processed for user `300a2341d082407a97458f4a144161af`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:48.114824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:48.181901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 917/1830] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:48.193287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:48.195069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:48.209004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:48.256845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed48062-67c8-4048-9bbe-848ee83f7112 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:48.260452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627d67e-509e-41a6-bd5b-900698e7c24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:48.261051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d67e-509e-41a6-bd5b-900698e7c24a service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:48.261327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d67e-509e-41a6-bd5b-900698e7c24a service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:48.261623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d67e-509e-41a6-bd5b-900698e7c24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:48.270254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627d67e-509e-41a6-bd5b-900698e7c24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:48.271706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627d67e-509e-41a6-bd5b-900698e7c24a service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:48.280601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 914/1831] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:52.550512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:52.663546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674c338-b7c3-419a-acee-ffa9928b41c6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:52.669870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:52.671031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:52.672174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:52.672792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] PATH_INFO: `/v3/users/300a2341d082407a97458f4a144161af` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:52.674285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] RBAC: Authorizing `identity:delete_user(user_id=300a2341d082407a97458f4a144161af)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:52.677116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:52.684328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:52.746547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300a2341d082407a97458f4a144161af'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:52.747470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300a2341d082407a97458f4a144161af'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:52.770064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300a2341d082407a97458f4a144161af'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:52.799535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300a2341d082407a97458f4a144161af'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:52.801561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300a2341d082407a97458f4a144161af'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:52.830906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 918/1832] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:52 2022] DELETE /identity/v3/users/300a2341d082407a97458f4a144161af => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:52.837961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:52.839235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:52.839692 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:52.895406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:52.896794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:52.974323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 915/1833] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:52.986246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:52.987899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:53.050590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168e450-335e-4106-ad7e-c82ce8dd8abb admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:53.054270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236d0b3-e1a0-4247-bd2b-40bd4d02e0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:53.054931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d0b3-e1a0-4247-bd2b-40bd4d02e0fc service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:53.055247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d0b3-e1a0-4247-bd2b-40bd4d02e0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:53.055661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236d0b3-e1a0-4247-bd2b-40bd4d02e0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:53.065818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236d0b3-e1a0-4247-bd2b-40bd4d02e0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:53.067294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236d0b3-e1a0-4247-bd2b-40bd4d02e0fc service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:53.077491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 919/1834] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:53.548491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:53.550019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:53.562736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a67f4-1d24-4638-8617-bcd1d8667f67 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:53.567526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:53.568160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:53.568444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:53.568720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] PATH_INFO: `/v3/projects/f50038fc43984b8e902487ac2e58e5e8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:53.575620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50038fc43984b8e902487ac2e58e5e8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:53.578207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:53.639203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50038fc43984b8e902487ac2e58e5e8', 'request_id': 'req-59741434-c976-49f0-81d0-71d3791f6186'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:53.642631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50038fc43984b8e902487ac2e58e5e8', 'request_id': 'req-59741434-c976-49f0-81d0-71d3791f6186'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:53.652396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50038fc43984b8e902487ac2e58e5e8', 'request_id': 'req-59741434-c976-49f0-81d0-71d3791f6186'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:53.670196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] The token cache is being invalidate because project f50038fc43984b8e902487ac2e58e5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:53.670553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:53.671244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-59741434-c976-49f0-81d0-71d3791f6186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:53.676679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 916/1835] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:53 2022] DELETE /identity/v3/projects/f50038fc43984b8e902487ac2e58e5e8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:59:53.682127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:53.682628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:53.682724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:53.720880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:53.722545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:53.792904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 920/1836] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:53.801149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:53.801449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:53.801731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:53.829388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:53.831542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:53.841399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 917/1837] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:53.846237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:53.857477 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690eab6-fde4-4b6e-9a51-4446333cc2dc None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:53.864521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:53.865290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:53.865676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:53.866056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:53.866951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:53.867644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:53.870442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:53.888976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 921/1838] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:59:53.895013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:53.917463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c24c40-4212-4e41-bb13-c495fd410373 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:53.925782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:53.926637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:53.926940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:53.927488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:53.928297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:53.930218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:53.965397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 918/1839] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:53 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:53.969222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:53.985850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0af22-7659-45a7-8494-07c7c8addf21 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:53.995890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:54.049821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.050436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.050723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.051027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.051675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.053583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.075979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 922/1840] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:53 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:54.080943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.091550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404598a-6d54-4f55-b60d-5dabe10b5d88 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.096694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.097333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.097635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.097927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.098618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.100260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.102295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:54.112748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 919/1841] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:54.116037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.126345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c620e-91c3-4a26-8b26-359ab81aeb73 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.130826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18a8a9-8fc7-48ae-b6cd-41a8e435d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.131446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18a8a9-8fc7-48ae-b6cd-41a8e435d027 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.131727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18a8a9-8fc7-48ae-b6cd-41a8e435d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.132003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18a8a9-8fc7-48ae-b6cd-41a8e435d027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.132637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18a8a9-8fc7-48ae-b6cd-41a8e435d027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.134215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18a8a9-8fc7-48ae-b6cd-41a8e435d027 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.145438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 923/1842] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:54.149959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.161539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3faf04-e351-42b3-af07-88d5b1adb291 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.167965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.169185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.169701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.170176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] PATH_INFO: `/v3/projects/2a2f9fa100c8410b97f5319bad236399/users/2ed5464d1b994b4ba63410739789cd9c/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.185126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2f9fa100c8410b97f5319bad236399, user_id=2ed5464d1b994b4ba63410739789cd9c, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.188532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.201578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 920/1843] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:54 2022] PUT /identity/v3/projects/2a2f9fa100c8410b97f5319bad236399/users/2ed5464d1b994b4ba63410739789cd9c/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:59:54.213738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.214022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.214297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.242725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] MFA Rules not processed for user `2ed5464d1b994b4ba63410739789cd9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:54.243958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:54.308726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 924/1844] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:54.313964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.398955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352410-a1ef-4018-bfde-4d7e22391922 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.405118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.405900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.406731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.407106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.407793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.409711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.440207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 921/1845] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:54 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:54.444191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.456505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58e4c-a19c-44e7-91fa-716d563df8c7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.520503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.521331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.521629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.521891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.522551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.524549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.558396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 925/1846] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:54 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:59:54.558538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.568801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bfe3a-6eed-4135-9b17-64cddecc62cb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.573747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.574344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.574625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.574932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.575568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.577158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.588783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 922/1847] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:54.592755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.603068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49af9c-b2ab-467c-baf8-4d6fc1c5c658 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.607690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aba752-59ee-446f-9411-08449b0440ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.608315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aba752-59ee-446f-9411-08449b0440ee admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.608598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aba752-59ee-446f-9411-08449b0440ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.609448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aba752-59ee-446f-9411-08449b0440ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.610237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aba752-59ee-446f-9411-08449b0440ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.611888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aba752-59ee-446f-9411-08449b0440ee admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.617046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:54.622755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 926/1848] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:54.627305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.637226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e0fc9-be3c-4b81-9ed9-d12e5e38fcc3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.641843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.642506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.642788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.643094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] PATH_INFO: `/v3/projects/3521e3f7d8e041ecb169e9da3cec0f84/users/de10a6549c0f4b6ea7f7bf404c191a6c/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.658177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3521e3f7d8e041ecb169e9da3cec0f84, user_id=de10a6549c0f4b6ea7f7bf404c191a6c, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.661209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.676202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 923/1849] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:54 2022] PUT /identity/v3/projects/3521e3f7d8e041ecb169e9da3cec0f84/users/de10a6549c0f4b6ea7f7bf404c191a6c/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:54.686741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.687965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.687965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.717403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] MFA Rules not processed for user `de10a6549c0f4b6ea7f7bf404c191a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:54.719314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:54.787328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 927/1850] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:54.791845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.861642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292b2f5-6154-4eea-9003-53f212e9b29c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.866288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.866924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.867199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.867467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.868097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.870021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:54.900082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 924/1851] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:54 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:54.901509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:54.912757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c51f5-c01c-4988-b283-7b7ca1437d92 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:54.980643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:54.981411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:54.981700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:54.981984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:54.982633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:54.985346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:55.010880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 928/1852] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:54 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:55.012847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:55.024286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673a929-0bad-43b6-ae4b-ddf014abbc54 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:55.032800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:55.033423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:55.033682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:55.033959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:55.034603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:55.036264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:55.045705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:55.058993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 925/1853] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:55.061063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:55.071777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e27a4-72b3-40c0-ba67-eb7d3ae2cc28 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:55.078948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:55.079891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:55.080249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:55.080606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] PATH_INFO: `/v3/projects/4f3f01c3a2fb47c7b0a5b60980b879f1/users/99f0f0d579a54a789a1f347090195647/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:55.098727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3f01c3a2fb47c7b0a5b60980b879f1, user_id=99f0f0d579a54a789a1f347090195647, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:55.101923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:55.119065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 929/1854] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:55 2022] PUT /identity/v3/projects/4f3f01c3a2fb47c7b0a5b60980b879f1/users/99f0f0d579a54a789a1f347090195647/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:55.122940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:55.133385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b27ff-5035-4c18-8bb4-2f50329f6b23 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:55.205063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b0d01-9bfa-4b60-a72d-d76823b6be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:55.205778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b0d01-9bfa-4b60-a72d-d76823b6be50 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:55.206134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b0d01-9bfa-4b60-a72d-d76823b6be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:55.206419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b0d01-9bfa-4b60-a72d-d76823b6be50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:55.207090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b0d01-9bfa-4b60-a72d-d76823b6be50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:55.208719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b0d01-9bfa-4b60-a72d-d76823b6be50 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:55.220665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 926/1855] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:55.224483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:55.236966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fac3f-1916-47f3-a10a-728ba46a6268 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:55.242146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:55.242806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:55.243112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:55.243396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] PATH_INFO: `/v3/projects/4f3f01c3a2fb47c7b0a5b60980b879f1/users/99f0f0d579a54a789a1f347090195647/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:55.245368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3f01c3a2fb47c7b0a5b60980b879f1, user_id=99f0f0d579a54a789a1f347090195647, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:55.248093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:55.258887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 930/1856] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:55 2022] PUT /identity/v3/projects/4f3f01c3a2fb47c7b0a5b60980b879f1/users/99f0f0d579a54a789a1f347090195647/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:55.270131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:55.270433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:55.270704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:55.303611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] MFA Rules not processed for user `99f0f0d579a54a789a1f347090195647`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:55.304208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:55.387008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 927/1857] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 17:59:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:55.403325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:55.404975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:55.453829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:55.466939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11117a14-d682-4809-9da4-2380e383a4e8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:55.471524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:55.472155 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:55.472329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:55.472607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:55.538098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:55.539664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:55.564232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 931/1858] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:57.257385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:57.258815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:57.273555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220eeb4-fced-4f92-959a-d27007ba5ee0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:57.285959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:57.286558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:57.286868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:57.287152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:57.381015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:57.382532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:57.394151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 928/1859] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:58.235659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:58.245792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e2111-d5ad-43e4-b65f-71fc7923c75e service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:58.249750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b20a74-17d1-481b-a4e4-bef1f8b131a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:58.250536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b20a74-17d1-481b-a4e4-bef1f8b131a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:58.250818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b20a74-17d1-481b-a4e4-bef1f8b131a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:58.251132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b20a74-17d1-481b-a4e4-bef1f8b131a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:58.261998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b20a74-17d1-481b-a4e4-bef1f8b131a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:58.263345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b20a74-17d1-481b-a4e4-bef1f8b131a7 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:58.282387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 932/1860] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:00.706850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:00.708798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:00.730939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:00.780458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff6392-6cff-4a49-995a-0363e4af32db service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:00.786009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:00.786719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:00.788431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:00.788431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] PATH_INFO: `/v3/users/5021cbef30f84af1a1cfcfde16aeeb54` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:00.789025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5021cbef30f84af1a1cfcfde16aeeb54)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:00.790979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:00.854404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5021cbef30f84af1a1cfcfde16aeeb54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:00.855466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5021cbef30f84af1a1cfcfde16aeeb54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:00.856273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5021cbef30f84af1a1cfcfde16aeeb54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:00.870597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5021cbef30f84af1a1cfcfde16aeeb54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:00.893009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5021cbef30f84af1a1cfcfde16aeeb54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:00.916278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 929/1861] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:00 2022] DELETE /identity/v3/users/5021cbef30f84af1a1cfcfde16aeeb54 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:00.922978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:00.923466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:00.923648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:00.951579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:00.953383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:01.040946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 933/1862] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:01.052452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:01.054140 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:01.118988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05caa9ac-0d18-4a9f-a2e3-6d9336bdbaff admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:01.126323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:01.127016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:01.127303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:01.127570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:01.137218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:01.138703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:01.149813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 930/1863] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:01.484512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:01.487405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:01.523044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83abc6-be11-440e-971a-de7eaff3f18b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:01.527972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:01.528650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:01.528934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:01.529212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] PATH_INFO: `/v3/projects/341ec0fef01c4c42ad0934ffe973fb8a` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:01.530476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=341ec0fef01c4c42ad0934ffe973fb8a)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:01.532992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:01.595429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '341ec0fef01c4c42ad0934ffe973fb8a', 'request_id': 'req-695621cd-cd6c-469e-a665-3523b434f13f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.612992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '341ec0fef01c4c42ad0934ffe973fb8a', 'request_id': 'req-695621cd-cd6c-469e-a665-3523b434f13f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.620636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '341ec0fef01c4c42ad0934ffe973fb8a', 'request_id': 'req-695621cd-cd6c-469e-a665-3523b434f13f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.656772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] The token cache is being invalidate because project 341ec0fef01c4c42ad0934ffe973fb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:01.657141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.657847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.662910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 934/1864] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:01 2022] DELETE /identity/v3/projects/341ec0fef01c4c42ad0934ffe973fb8a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:01.664989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:01.684812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e144ee-413c-444c-aca0-b52dce34d9a3 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:01.747226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:01.778246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:01.779017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:01.779401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:01.779755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] PATH_INFO: `/v3/users/5ab1de7bf92e4aa3988050a2e8a3f77a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:01.781350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab1de7bf92e4aa3988050a2e8a3f77a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:01.783760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:01.848758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab1de7bf92e4aa3988050a2e8a3f77a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.850667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab1de7bf92e4aa3988050a2e8a3f77a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.851849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab1de7bf92e4aa3988050a2e8a3f77a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.865175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab1de7bf92e4aa3988050a2e8a3f77a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.880555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab1de7bf92e4aa3988050a2e8a3f77a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:01.916283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 931/1865] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:01 2022] DELETE /identity/v3/users/5ab1de7bf92e4aa3988050a2e8a3f77a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:02.337322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:02.338985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:02.360162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:02.367122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695621cd-cd6c-469e-a665-3523b434f13f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:02.371606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:02.388057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:02.489264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:02.490103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:02.491051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:02.491433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] PATH_INFO: `/v3/projects/4447b9c7cc454fd7b29324a5038cd98e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:02.493537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4447b9c7cc454fd7b29324a5038cd98e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:02.517718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006b967-2d61-4cd0-9c87-914b4127f67d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:02.521778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:02.524165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:02.525053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:02.525672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:02.526132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] PATH_INFO: `/v3/users/2ed5464d1b994b4ba63410739789cd9c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:02.527544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed5464d1b994b4ba63410739789cd9c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:02.542888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:02.654065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed5464d1b994b4ba63410739789cd9c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.654861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed5464d1b994b4ba63410739789cd9c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.661251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed5464d1b994b4ba63410739789cd9c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.666938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4447b9c7cc454fd7b29324a5038cd98e', 'request_id': 'req-f2af5702-3955-4a38-a27e-2bc30c847b06'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.675509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4447b9c7cc454fd7b29324a5038cd98e', 'request_id': 'req-f2af5702-3955-4a38-a27e-2bc30c847b06'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.683046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed5464d1b994b4ba63410739789cd9c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.690639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4447b9c7cc454fd7b29324a5038cd98e', 'request_id': 'req-f2af5702-3955-4a38-a27e-2bc30c847b06'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.691437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:02.707275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88962095-71a2-4a9b-bbbb-09683a5e20f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed5464d1b994b4ba63410739789cd9c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.730207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] The token cache is being invalidate because project 4447b9c7cc454fd7b29324a5038cd98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:02.731424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.733710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f2af5702-3955-4a38-a27e-2bc30c847b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:02.738648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 932/1867] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:02 2022] DELETE /identity/v3/users/2ed5464d1b994b4ba63410739789cd9c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:02.740414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 935/1867] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:02 2022] DELETE /identity/v3/projects/4447b9c7cc454fd7b29324a5038cd98e => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:02.740926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:02.741508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:02.742145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:02.745402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e84a8-a1c6-402a-a864-9e539e69c65d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:02.745402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e84a8-a1c6-402a-a864-9e539e69c65d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:02.745402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e84a8-a1c6-402a-a864-9e539e69c65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:02.768299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7c6e84a8-a1c6-402a-a864-9e539e69c65d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:02.769733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7c6e84a8-a1c6-402a-a864-9e539e69c65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:02.795089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:02.797222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:02.887013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 933/1868] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:02.888653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:02.892176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:02.903166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 936/1869] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:02.909994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:02.910335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:02.910627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:02.913821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1452d-18b3-4b64-9ac9-77b66e8c920d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:02.926525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:02.926525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:02.926525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:02.926525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] PATH_INFO: `/v3/users/b6a6e9ed617042d2bf4c059356e36325` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:02.926919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a6e9ed617042d2bf4c059356e36325)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:02.926919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:02.948869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:02.950365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:02.963172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 937/1870] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:02.983147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:02.985862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.012521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a6e9ed617042d2bf4c059356e36325'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.013328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a6e9ed617042d2bf4c059356e36325'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.014068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a6e9ed617042d2bf4c059356e36325'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.023616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a6e9ed617042d2bf4c059356e36325'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.032883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:03.038177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a6e9ed617042d2bf4c059356e36325'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.059747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 934/1871] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:02 2022] DELETE /identity/v3/users/b6a6e9ed617042d2bf4c059356e36325 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:03.059747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.059955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.067033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da63abc-d8bd-415e-be9c-23bb0b6e2724 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.141300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:03.142819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.143645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.144095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.144379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.146602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.161577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba834d8e-68e1-4e9a-8bed-911b245e3a64 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.163216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.164651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4902a-f7a0-4d63-b62b-50df1be4644b service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.167290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.167988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.168341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.168819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.169545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:03.169922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.172103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.180009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 938/1872] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:03.181360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.181686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.181904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.197713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 935/1873] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:03.200467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.221076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7141c7d-5537-46cc-ba27-ee574cc88c1c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.226624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.226935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:03.227398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.227679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.227957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.228076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.228742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.230847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.236790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 939/1874] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:03.248882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.257479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.270002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 936/1875] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:03 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:03.274115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.280253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d03001-e2e2-46b6-9536-fb6202f054a0 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.337486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.338301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.338723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.339119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.355551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444a592-9634-4db9-bb2f-1883d53770b4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.363571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.364210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.364492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.364766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.365413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.367844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.402887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 937/1876] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:03 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:03.407096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.417841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a53dc6-f6af-4d2e-8d10-3213fbb211d0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.424407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.425028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.425308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.425584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.426287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.428130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.432839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.434543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.439858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 938/1877] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:03.443676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.448936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 940/1878] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:03.454280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262e17e-5dfc-444a-beb7-1635a0f6cf85 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.458955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962aef9-6bba-41fc-87ff-ca2a092a8c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.459627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962aef9-6bba-41fc-87ff-ca2a092a8c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.460026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962aef9-6bba-41fc-87ff-ca2a092a8c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.460405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962aef9-6bba-41fc-87ff-ca2a092a8c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.461104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962aef9-6bba-41fc-87ff-ca2a092a8c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.462755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962aef9-6bba-41fc-87ff-ca2a092a8c0a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.473612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 939/1879] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:03.477795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.488230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993de4f-eee3-432b-b1a9-9ef97060211e service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.492903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.493556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.493836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.494110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] PATH_INFO: `/v3/projects/f963631f24e843bab34ce5ef8f02618f/users/7959af32c6944ba4958ffa401381a43f/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.509973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f963631f24e843bab34ce5ef8f02618f, user_id=7959af32c6944ba4958ffa401381a43f, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.512652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.527022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 941/1880] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:03 2022] PUT /identity/v3/projects/f963631f24e843bab34ce5ef8f02618f/users/7959af32c6944ba4958ffa401381a43f/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:03.538522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.538815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.539153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.570906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] MFA Rules not processed for user `7959af32c6944ba4958ffa401381a43f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:03.572301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.628625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.630405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.642357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:03.648143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 940/1881] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:03.652393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.678031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:03.738750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6c4fa-5b92-410e-be3e-1fece285de67 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.744391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.745158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.745620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.746011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.746883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.748824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.755307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea8c48-3234-4aa7-90df-1274fc784674 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.760121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.760769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.761030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.761307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] PATH_INFO: `/v3/projects/2a2f9fa100c8410b97f5319bad236399` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.762534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2f9fa100c8410b97f5319bad236399)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.764571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.779765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 941/1882] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:03 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:03.784167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.837930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2f9fa100c8410b97f5319bad236399', 'request_id': 'req-8cb14604-ec85-44bb-8c04-645639ca7765'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.853109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2f9fa100c8410b97f5319bad236399', 'request_id': 'req-8cb14604-ec85-44bb-8c04-645639ca7765'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.860791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2f9fa100c8410b97f5319bad236399', 'request_id': 'req-8cb14604-ec85-44bb-8c04-645639ca7765'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.880738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d030-fcb8-467a-b8fc-38e3c6f1f78c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:03.886803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] The token cache is being invalidate because project 2a2f9fa100c8410b97f5319bad236399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:03.888743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.890357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:03.893075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 942/1883] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:03 2022] DELETE /identity/v3/projects/2a2f9fa100c8410b97f5319bad236399 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:03.893752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:03.895231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:03.955192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:03.955905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:03.956288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:03.956567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:03.957291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:03.959470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:03.995653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14604-ec85-44bb-8c04-645639ca7765 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.002683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.003596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.004008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.004345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] PATH_INFO: `/v3/projects/0ac80b86ab4f4d14bdc0c53e079b5213` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.005449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac80b86ab4f4d14bdc0c53e079b5213)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.007439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.009203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 942/1884] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:03 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:04.010560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.012501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.027178 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4f02d-ea74-4c12-bd29-5e57a1cc3b2a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.033334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.034271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.034745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.035149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] PATH_INFO: `/v3/users/de10a6549c0f4b6ea7f7bf404c191a6c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.036662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de10a6549c0f4b6ea7f7bf404c191a6c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.039017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.087330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:04.100697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de10a6549c0f4b6ea7f7bf404c191a6c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.102255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:04.103260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de10a6549c0f4b6ea7f7bf404c191a6c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.103935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de10a6549c0f4b6ea7f7bf404c191a6c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.120619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de10a6549c0f4b6ea7f7bf404c191a6c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.125333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac80b86ab4f4d14bdc0c53e079b5213', 'request_id': 'req-d0974163-3038-4405-bf7b-3c7cf260261e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.138406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac80b86ab4f4d14bdc0c53e079b5213', 'request_id': 'req-d0974163-3038-4405-bf7b-3c7cf260261e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.139149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de10a6549c0f4b6ea7f7bf404c191a6c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.149749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac80b86ab4f4d14bdc0c53e079b5213', 'request_id': 'req-d0974163-3038-4405-bf7b-3c7cf260261e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.158129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 943/1885] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:04 2022] DELETE /identity/v3/users/de10a6549c0f4b6ea7f7bf404c191a6c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:04.161094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.161094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.179122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] The token cache is being invalidate because project 0ac80b86ab4f4d14bdc0c53e079b5213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:04.179957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.180938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.186476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 943/1886] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:03 2022] DELETE /identity/v3/projects/0ac80b86ab4f4d14bdc0c53e079b5213 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:04.187831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.189456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.329148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0974163-3038-4405-bf7b-3c7cf260261e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.342368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255e10-f5b9-4b89-8480-6935552ba63d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.342547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.344872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.345236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.345586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] PATH_INFO: `/v3/users/d0021a9073d6489f961bdad5cc053e6d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.347182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0021a9073d6489f961bdad5cc053e6d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.349632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.356685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.357468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.357819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.358366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.374164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.381084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.407315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 944/1887] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:04.411846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.415748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.442017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89698f4-74e1-4d4c-af9f-0a55d4ea98f2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.459341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.459824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.460105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.460374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] PATH_INFO: `/v3/projects/9bf855c989214ca38f8df54d4d8f1013/users/ffc4602c7420488cb93a95d15467f7a5/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.478577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0021a9073d6489f961bdad5cc053e6d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.487007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0021a9073d6489f961bdad5cc053e6d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.501739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0021a9073d6489f961bdad5cc053e6d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.504052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf855c989214ca38f8df54d4d8f1013, user_id=ffc4602c7420488cb93a95d15467f7a5, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.511025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.530386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0021a9073d6489f961bdad5cc053e6d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.533118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0021a9073d6489f961bdad5cc053e6d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.541213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 945/1888] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:04 2022] PUT /identity/v3/projects/9bf855c989214ca38f8df54d4d8f1013/users/ffc4602c7420488cb93a95d15467f7a5/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:04.542770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.547429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.554101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 944/1889] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:04 2022] DELETE /identity/v3/users/d0021a9073d6489f961bdad5cc053e6d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:04.554751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.556296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.616766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:04.651065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab6ced-2a64-461c-877d-c9f867f24232 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.664831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.665598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.665877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.666382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.667093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.668943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.684025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 945/1890] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:04.692725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.692897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40540771-c3c6-4641-8e74-ecf2e5696ee6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.700142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.700865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.701064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.701389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] PATH_INFO: `/v3/projects/3521e3f7d8e041ecb169e9da3cec0f84` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.703466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=3521e3f7d8e041ecb169e9da3cec0f84)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.705406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.715651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ae288-1269-42ba-83aa-2a48884416cf admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:04.732296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacf948-3979-420f-b34a-ebc191c6b67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:04.732990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacf948-3979-420f-b34a-ebc191c6b67c admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.733277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacf948-3979-420f-b34a-ebc191c6b67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.733550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacf948-3979-420f-b34a-ebc191c6b67c admin admin] PATH_INFO: `/v3/projects/9bf855c989214ca38f8df54d4d8f1013/users/ffc4602c7420488cb93a95d15467f7a5/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.741821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacf948-3979-420f-b34a-ebc191c6b67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf855c989214ca38f8df54d4d8f1013, user_id=ffc4602c7420488cb93a95d15467f7a5, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:04.744522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacf948-3979-420f-b34a-ebc191c6b67c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:04.774661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 946/1891] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:04 2022] PUT /identity/v3/projects/9bf855c989214ca38f8df54d4d8f1013/users/ffc4602c7420488cb93a95d15467f7a5/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:04.775444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:04.776833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:04.777169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:04.777402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:04.809472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3521e3f7d8e041ecb169e9da3cec0f84', 'request_id': 'req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.823274 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3521e3f7d8e041ecb169e9da3cec0f84', 'request_id': 'req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.844509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] MFA Rules not processed for user `ffc4602c7420488cb93a95d15467f7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:04.846115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.846467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3521e3f7d8e041ecb169e9da3cec0f84', 'request_id': 'req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.868815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] The token cache is being invalidate because project 3521e3f7d8e041ecb169e9da3cec0f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:04.869112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.882144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:04.887945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 946/1892] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:04 2022] DELETE /identity/v3/projects/3521e3f7d8e041ecb169e9da3cec0f84 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:04.888935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.895564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:04.971920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 947/1893] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:04.993326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:04.994883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.005146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4fe7-cb45-4083-bd87-5ac6e686f1db admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:05.011702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:05.012455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.012808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.013150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] PATH_INFO: `/v3/users/99f0f0d579a54a789a1f347090195647` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.016129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f0f0d579a54a789a1f347090195647)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:05.018180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:05.081117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f0f0d579a54a789a1f347090195647'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.081871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f0f0d579a54a789a1f347090195647'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.082548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f0f0d579a54a789a1f347090195647'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.086021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed900453-0ec2-4772-8b30-93b71b4970f9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:05.090199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f0f0d579a54a789a1f347090195647'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.091409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:05.092281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.092525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.092801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.095405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.104792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f0f0d579a54a789a1f347090195647'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.119396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:05.121950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f1dbc-7c51-49a9-8e86-60d09ff8868b service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:05.123752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 947/1894] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:04 2022] DELETE /identity/v3/users/99f0f0d579a54a789a1f347090195647 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:05.127876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:05.130252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.155212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 948/1895] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:00:05.183748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:05.231099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6f6de-f8e3-4a5a-882d-c4af4a7a7092 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:05.248587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:05.250871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.251230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.251712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] PATH_INFO: `/v3/projects/a4a2ca7daf6b44d9a76b6c4f6ca9658b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.253154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a2ca7daf6b44d9a76b6c4f6ca9658b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:05.255263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:05.359996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a2ca7daf6b44d9a76b6c4f6ca9658b', 'request_id': 'req-8a002124-02ae-4afb-9ad5-708734d8e86f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.371463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a2ca7daf6b44d9a76b6c4f6ca9658b', 'request_id': 'req-8a002124-02ae-4afb-9ad5-708734d8e86f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.389375 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a2ca7daf6b44d9a76b6c4f6ca9658b', 'request_id': 'req-8a002124-02ae-4afb-9ad5-708734d8e86f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.411349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] The token cache is being invalidate because project a4a2ca7daf6b44d9a76b6c4f6ca9658b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:05.411916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.413924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8a002124-02ae-4afb-9ad5-708734d8e86f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.416108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 948/1896] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:05 2022] DELETE /identity/v3/projects/a4a2ca7daf6b44d9a76b6c4f6ca9658b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:05.422688 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.423239 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.423943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.464790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:05.465799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.572455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:05.597325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 949/1897] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:05.597769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.598186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.598703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.657004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:05.660191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.684120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 949/1898] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:05.691069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:05.708730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55427e-c4cc-4853-ad39-71862cb32ea3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:05.717545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:05.719365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.729080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:05.736533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.736926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.737312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.739760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:05.740746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:05.745390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:05.746704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd0127-7858-47ee-a950-dfabb5c70569 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:05.754195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:05.754876 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.755168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.755446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] PATH_INFO: `/v3/projects/4f3f01c3a2fb47c7b0a5b60980b879f1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.756770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3f01c3a2fb47c7b0a5b60980b879f1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:05.759457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:05.782912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 950/1899] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:05.786276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:05.798581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cae8c-e515-48a5-bee3-34626b0b60d7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:05.804851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:05.805577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:05.805959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:05.806347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:05.807273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:05.810167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:05.858534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3f01c3a2fb47c7b0a5b60980b879f1', 'request_id': 'req-67226661-772a-4fdb-b31a-e0a9f76732a2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.868107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3f01c3a2fb47c7b0a5b60980b879f1', 'request_id': 'req-67226661-772a-4fdb-b31a-e0a9f76732a2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.870693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 951/1900] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:05 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:05.871888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:05.874457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:05.886689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3f01c3a2fb47c7b0a5b60980b879f1', 'request_id': 'req-67226661-772a-4fdb-b31a-e0a9f76732a2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.925834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] The token cache is being invalidate because project 4f3f01c3a2fb47c7b0a5b60980b879f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:05.927057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.939936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:05.946524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 950/1901] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:05 2022] DELETE /identity/v3/projects/4f3f01c3a2fb47c7b0a5b60980b879f1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:05.946524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:05.948797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.016254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958633-4f99-4f22-8552-6a808d0176b0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.021381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:06.021611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4602c7420488cb93a95d15467f7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf855c989214ca38f8df54d4d8f1013', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.022339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.022679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.023047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.023971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:06.024274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.026107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4566-12b4-49b2-8e14-a091d785dc40 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.043127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 952/1902] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:06.043127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.043127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.043127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.050357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226661-772a-4fdb-b31a-e0a9f76732a2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.065030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:06.066281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.119844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932bc03-c775-423d-8c9d-da7448e2a5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.120574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932bc03-c775-423d-8c9d-da7448e2a5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.120860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932bc03-c775-423d-8c9d-da7448e2a5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.121001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932bc03-c775-423d-8c9d-da7448e2a5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.121629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932bc03-c775-423d-8c9d-da7448e2a5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.123864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932bc03-c775-423d-8c9d-da7448e2a5f2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.147147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 953/1903] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:06.148261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.149744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.165699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ba487-4d8f-47af-ba52-206768ff6a9b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.170354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4602c7420488cb93a95d15467f7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf855c989214ca38f8df54d4d8f1013', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.172145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 951/1904] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:05 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:06.172351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.172351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.172601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.173215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.173335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.173513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.173792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.175322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.193163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:06.194376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.204145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 952/1905] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:06.205175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.212546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 954/1906] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/projects => generated 354 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:06.212546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.215127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bc082-3aac-420a-8a3f-763e3e61a61d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.254740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:06.273719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.274390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.274671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.274984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.275624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.277247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.278294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f95f7a-dfa4-4aac-a64d-174605cc56d8 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.283724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.284333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.284614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.284890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.285536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:06.285847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.287460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.288452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 953/1907] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:06.289265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.305647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 955/1908] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:06.305647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.312880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6da5c6-960b-4aa2-a16a-a51185e437d0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.317515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.318177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.318526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.318884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.319526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.321101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.339049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 956/1909] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:06.339864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.350420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b03984-3548-453c-8cb8-2a9ca8a5d072 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.355291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.355882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.356158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.356429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.357044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.357784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39183ff8-3669-4539-adad-81e4af604cdd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.359024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.362549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4602c7420488cb93a95d15467f7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf855c989214ca38f8df54d4d8f1013', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.363189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.363578 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.363752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.364406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:06.364718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.366407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.378719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 954/1910] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:06.380163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.386605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:06.392603 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 957/1911] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:06.393310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3e74-8888-4b14-abe6-fa26a23c2a4d tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.393780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.459524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43078c72-3994-43d0-9544-b0d7beb9bd2c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.460171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.460284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.460503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.460769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] PATH_INFO: `/v3/projects/471048704f26423d8b157c0b2effbfe6/users/0d3e8cbc685b4be1acbf2fd0612fa94e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.464268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.464877 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.465163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.465445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.466080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.468039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.475980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] RBAC: Authorizing `identity:create_grant(project_id=471048704f26423d8b157c0b2effbfe6, user_id=0d3e8cbc685b4be1acbf2fd0612fa94e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.478673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.492077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 955/1912] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:06 2022] PUT /identity/v3/projects/471048704f26423d8b157c0b2effbfe6/users/0d3e8cbc685b4be1acbf2fd0612fa94e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:06.492347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 958/1913] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:06.493044 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.496192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.506496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94382175-97e0-432d-a704-e0bd22b75dcb admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.561936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19324d56-6c7c-4c6f-87be-1418e0d5ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.562544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324d56-6c7c-4c6f-87be-1418e0d5ddac admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.562821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324d56-6c7c-4c6f-87be-1418e0d5ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.563118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19324d56-6c7c-4c6f-87be-1418e0d5ddac admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.563779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19324d56-6c7c-4c6f-87be-1418e0d5ddac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.565359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19324d56-6c7c-4c6f-87be-1418e0d5ddac admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.566724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9618df2-bacb-4fca-8377-f89ac0b1620a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.571743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc4602c7420488cb93a95d15467f7a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf855c989214ca38f8df54d4d8f1013', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.572525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.573063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.573338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] PATH_INFO: `/v3/projects/d3f8226a98d04c879ad8cf601bd805ea` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.574464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d3f8226a98d04c879ad8cf601bd805ea)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.575317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 959/1914] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:06.575317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.579207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.579303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.579303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.604883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] MFA Rules not processed for user `0d3e8cbc685b4be1acbf2fd0612fa94e`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:06.606096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.619119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:06.630261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f8226a98d04c879ad8cf601bd805ea', 'request_id': 'req-45abfee1-a83b-446e-99a1-6a986cbb0512'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:06.640583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f8226a98d04c879ad8cf601bd805ea', 'request_id': 'req-45abfee1-a83b-446e-99a1-6a986cbb0512'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:06.650979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f8226a98d04c879ad8cf601bd805ea', 'request_id': 'req-45abfee1-a83b-446e-99a1-6a986cbb0512'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:06.664791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] The token cache is being invalidate because project d3f8226a98d04c879ad8cf601bd805ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:06.665082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:06.665845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:06.667385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 956/1915] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:06 2022] DELETE /identity/v3/projects/d3f8226a98d04c879ad8cf601bd805ea => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:06.668318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.669773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.683172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 960/1916] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:06.688519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.690144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:06.706069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5514a3-136f-47f2-846f-fb23083c76c7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.707794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:06.773174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abfee1-a83b-446e-99a1-6a986cbb0512 tempest-VolumeQuotaClassesTest-1002960952 tempest-VolumeQuotaClassesTest-1002960952-project-admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.779487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.780170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.780454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.781021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.781764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.783469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.800418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 957/1917] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:06.807115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.826944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f26ee-fa2b-4cda-b0db-2fac9fd48e10 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:06.833590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.834363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.834732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.834827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1c02e-a6f8-48b6-b3f6-c6f1ba48e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:06.835286 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.836081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.836209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1c02e-a6f8-48b6-b3f6-c6f1ba48e7be admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.836438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1c02e-a6f8-48b6-b3f6-c6f1ba48e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.836824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1c02e-a6f8-48b6-b3f6-c6f1ba48e7be admin admin] PATH_INFO: `/v3/projects/33baca214d8f4909aa19736491569b30/users/c655248e25a14d9dbe4d5c491b688991/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.838494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.857064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1c02e-a6f8-48b6-b3f6-c6f1ba48e7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=33baca214d8f4909aa19736491569b30, user_id=c655248e25a14d9dbe4d5c491b688991, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:06.863821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1c02e-a6f8-48b6-b3f6-c6f1ba48e7be admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:06.880850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 961/1918] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/projects => generated 361 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:06.883873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 958/1919] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:06 2022] PUT /identity/v3/projects/33baca214d8f4909aa19736491569b30/users/c655248e25a14d9dbe4d5c491b688991/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:06.884851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:06.895714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:06.896237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:06.896237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:06.933334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] MFA Rules not processed for user `c655248e25a14d9dbe4d5c491b688991`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:06.935760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.017602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9496b6-8039-4fa4-97ef-662b27ecf627 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.024733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 959/1920] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:07.027281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.027906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.028183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.028453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.029086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.030704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.031319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.031921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.049385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b71fa4-92d1-4c79-ae9f-9a00bb63bbf5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.056670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.057425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.057812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 962/1921] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:06 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:07.058793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.060850 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.061341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] PATH_INFO: `/v3/users/7959af32c6944ba4958ffa401381a43f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.062444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.063486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7959af32c6944ba4958ffa401381a43f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.065469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.108639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:07.128320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7959af32c6944ba4958ffa401381a43f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:07.129476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7959af32c6944ba4958ffa401381a43f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:07.130135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7959af32c6944ba4958ffa401381a43f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:07.137767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7959af32c6944ba4958ffa401381a43f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:07.160240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:07.162385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7959af32c6944ba4958ffa401381a43f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:07.168741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277ca3-0ed5-4472-b7f2-e063bcaa8432 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.174375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.175032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.176596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.176948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.178957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.191329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 960/1922] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:07 2022] DELETE /identity/v3/users/7959af32c6944ba4958ffa401381a43f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:07.192518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.195177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.198740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.200229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292c6f1-0af7-4096-8c4f-369f9dc9665d service glance] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.212404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 963/1923] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:07.214601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.214913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.215467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.258594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:07.259527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.286495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904dbdd-e456-43a3-bf37-70a0e7daaea9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.292962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.294127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.294413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.294690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.295383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.298540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.315127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 961/1924] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:07.317476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.336967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726473b-0528-4e96-9395-cc7a27e471e8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.341881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.342677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.343125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.343735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] PATH_INFO: `/v3/projects/11171fc97ffa482fb91550e85271979c/users/f7f350aaec6443338e7fb01c4b9c0c60/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.353268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 964/1925] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:07.368017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.369193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11171fc97ffa482fb91550e85271979c, user_id=f7f350aaec6443338e7fb01c4b9c0c60, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.370862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.372417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.395002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 962/1926] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:07 2022] PUT /identity/v3/projects/11171fc97ffa482fb91550e85271979c/users/f7f350aaec6443338e7fb01c4b9c0c60/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:07.396126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.484537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217f3ac-eee3-405d-afff-62f40d6d72ec None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.492539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ff1f2-ad69-4701-a4ba-17269ee70f26 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.498379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.499384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.500169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.500617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.501522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.503960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.515905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 963/1927] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:07.523062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.534755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4689e9-c40c-49e6-a567-3367cf9d9027 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:07.539935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894b4cd-8616-4389-972a-50817dd2e737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.540659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b4cd-8616-4389-972a-50817dd2e737 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.541057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b4cd-8616-4389-972a-50817dd2e737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.542079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894b4cd-8616-4389-972a-50817dd2e737 admin admin] PATH_INFO: `/v3/projects/11171fc97ffa482fb91550e85271979c/users/f7f350aaec6443338e7fb01c4b9c0c60/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.542475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:07.543119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.543393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.543668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.544045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894b4cd-8616-4389-972a-50817dd2e737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11171fc97ffa482fb91550e85271979c, user_id=f7f350aaec6443338e7fb01c4b9c0c60, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.547449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894b4cd-8616-4389-972a-50817dd2e737 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.556988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:07.558438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:07.559473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 964/1928] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:07 2022] PUT /identity/v3/projects/11171fc97ffa482fb91550e85271979c/users/f7f350aaec6443338e7fb01c4b9c0c60/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:07.570045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:07.570207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 965/1929] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:07.576076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:07.576520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:07.615786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] MFA Rules not processed for user `f7f350aaec6443338e7fb01c4b9c0c60`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:07.617655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.661723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:07.786525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 965/1930] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:07.797196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:07.798511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:07.830370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:07.936350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ae0cc-2a04-4175-98ab-1067c380fa06 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:08.028086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:08.029704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:08.047170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:08.047808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:08.048176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:08.048353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:08.074178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:08.085523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:08.108926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 966/1931] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:00:08.165623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec927e-de79-42f9-88a2-0d6d2684139f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:08.178036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:08.178713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:08.179053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:08.179337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] PATH_INFO: `/v3/projects/f963631f24e843bab34ce5ef8f02618f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:08.180574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f963631f24e843bab34ce5ef8f02618f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:08.182528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:08.268528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f963631f24e843bab34ce5ef8f02618f', 'request_id': 'req-2f4a6036-b258-4cfc-845a-8fd8230af467'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.273673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:08.278430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f963631f24e843bab34ce5ef8f02618f', 'request_id': 'req-2f4a6036-b258-4cfc-845a-8fd8230af467'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.290286 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85a619-f08e-4852-a30b-539a41524692 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:08.293378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f963631f24e843bab34ce5ef8f02618f', 'request_id': 'req-2f4a6036-b258-4cfc-845a-8fd8230af467'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.308937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] The token cache is being invalidate because project f963631f24e843bab34ce5ef8f02618f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:08.309538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.310405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.313832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 966/1932] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:08 2022] DELETE /identity/v3/projects/f963631f24e843bab34ce5ef8f02618f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:08.317764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:08.319320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:08.379491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:08.380402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:08.380844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:08.381261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:08.383328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:08.423743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a6036-b258-4cfc-845a-8fd8230af467 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:08.430630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:08.431609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:08.432905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:08.433670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] PATH_INFO: `/v3/users/ffc4602c7420488cb93a95d15467f7a5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:08.435808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc4602c7420488cb93a95d15467f7a5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:08.439290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:08.443169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:08.542269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:08.545783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:08.557079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc4602c7420488cb93a95d15467f7a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.557888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc4602c7420488cb93a95d15467f7a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.558561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc4602c7420488cb93a95d15467f7a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.566222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 967/1933] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:08.569029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc4602c7420488cb93a95d15467f7a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.587280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-21ea1119-f002-4656-aa2c-10458ac3d71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc4602c7420488cb93a95d15467f7a5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:08.615769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 967/1934] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:08 2022] DELETE /identity/v3/users/ffc4602c7420488cb93a95d15467f7a5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:09.024370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.040720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa80ae-ec26-4376-88d9-7c5ebb50f2cf service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.065726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:09.112721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.113402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.113687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.113991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] PATH_INFO: `/v3/projects/9bf855c989214ca38f8df54d4d8f1013` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.115424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf855c989214ca38f8df54d4d8f1013)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.117232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.169716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf855c989214ca38f8df54d4d8f1013', 'request_id': 'req-5164a913-dae7-46e0-8f8d-b789c191b993'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:09.177080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf855c989214ca38f8df54d4d8f1013', 'request_id': 'req-5164a913-dae7-46e0-8f8d-b789c191b993'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:09.184209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf855c989214ca38f8df54d4d8f1013', 'request_id': 'req-5164a913-dae7-46e0-8f8d-b789c191b993'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:09.199953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] The token cache is being invalidate because project 9bf855c989214ca38f8df54d4d8f1013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:09.200265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:09.201039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5164a913-dae7-46e0-8f8d-b789c191b993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:09.202944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 968/1935] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:09 2022] DELETE /identity/v3/projects/9bf855c989214ca38f8df54d4d8f1013 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:09.207996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.208580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.208580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.241198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:09.242476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:09.324515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 968/1936] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:09.333768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.334082 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.334363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.362263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:09.363548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:09.378581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 969/1937] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:09.380237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.392589 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3860c-6bd5-447c-aacc-52147cde68ee None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.400849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.401557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.402428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.402806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.403571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:09.403950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.406888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.419450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 969/1938] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:09.424833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.439992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbd5c0-2814-4f06-9d2b-6964b4211225 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.446616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.447764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.447764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.448294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.448722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.450819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.492539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.492993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 970/1939] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:09 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:09.529610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbef9b-297d-489b-9697-66510250278a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.663263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.664570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.664962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.665382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.666126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.668974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.686241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:09.722694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 970/1940] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:09 2022] POST /identity/v3/users => generated 294 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:09.723670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.735258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c951399-168c-4654-ac77-0366023cc17b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.740650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.741277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.741557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.741832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.742553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.744188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.756913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 971/1941] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:09.764170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.774720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46f323-bc5d-4575-9cd9-02f803a8eb1a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.779478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ff48d-d878-4d09-b845-231907042dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.780207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ff48d-d878-4d09-b845-231907042dea admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.780493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ff48d-d878-4d09-b845-231907042dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.781445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ff48d-d878-4d09-b845-231907042dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.781946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ff48d-d878-4d09-b845-231907042dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.783788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ff48d-d878-4d09-b845-231907042dea admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.799885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 971/1942] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:09.804698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:09.817427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3baedc-cd06-4aad-8ef2-abbcaffa5b9b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:09.824838 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:09.825511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.826136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.826810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] PATH_INFO: `/v3/projects/0eae974749644160b8e94e9efe8026b0/users/7882fd175a5a41bbbfa43fdfac7e497f/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.846565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eae974749644160b8e94e9efe8026b0, user_id=7882fd175a5a41bbbfa43fdfac7e497f, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:09.849462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:09.871379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 972/1943] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:09 2022] PUT /identity/v3/projects/0eae974749644160b8e94e9efe8026b0/users/7882fd175a5a41bbbfa43fdfac7e497f/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:09.882965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:09.883303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:09.883577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:09.925254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] MFA Rules not processed for user `7882fd175a5a41bbbfa43fdfac7e497f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:09.926552 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:10.073563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 972/1944] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:10.075271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:10.172651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:10.191544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d07f56-f55d-40dd-a72a-2f31541e3929 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:10.204621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:10.205350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.205815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.206098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.206760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:10.209471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:10.247133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 973/1945] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:10 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:10.251731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:10.267019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617db81b-cae8-4b0a-9c74-a420a6cb9ebf None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:10.396904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:10.398396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.399103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.399622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.403986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:10.408375 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:10.450522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 973/1946] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:10 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:10.455188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:10.469909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606623f2-26b3-4bd1-81a9-e0ad5c2bbdc8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:10.475812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:10.476689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.477198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.477664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.478529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:10.480192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:10.508213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 974/1947] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:10.511441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:10.523060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4dc75-771f-4015-a0cb-556b2f825b1b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:10.536059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b8461-a16e-43b8-9f0d-556bb9ba8a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:10.537224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b8461-a16e-43b8-9f0d-556bb9ba8a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.538341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b8461-a16e-43b8-9f0d-556bb9ba8a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.538893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b8461-a16e-43b8-9f0d-556bb9ba8a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.539860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b8461-a16e-43b8-9f0d-556bb9ba8a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:10.541681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b8461-a16e-43b8-9f0d-556bb9ba8a0c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:10.555332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 974/1948] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:10.560648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:10.579735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4fa54-4d1a-4bd5-a493-d8387b03cd4d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:10.586018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:10.587913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.588607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.589530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] PATH_INFO: `/v3/projects/da5b167f189c4aa1b8f89805681a626b/users/80cbac74fb874b3f816a354fad776632/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.654078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5b167f189c4aa1b8f89805681a626b, user_id=80cbac74fb874b3f816a354fad776632, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:10.657912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:10.677638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 975/1949] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:10 2022] PUT /identity/v3/projects/da5b167f189c4aa1b8f89805681a626b/users/80cbac74fb874b3f816a354fad776632/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:10.688161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.688752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.689027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.730599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] MFA Rules not processed for user `80cbac74fb874b3f816a354fad776632`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:10.732860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:10.817023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:10.867962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 975/1950] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:10.873010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:10.973513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbc0d4-60ef-4569-8251-c72355329c64 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:10.995203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:10.995203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:10.995203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:10.995203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:10.996073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:10.999919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.051599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 976/1951] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:10 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:11.055432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:11.072497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23959d8-7d8a-4b3a-94ef-cb6a176465fe None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:11.165100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:11.165728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.166056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.166337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.167432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:11.171188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.207406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 976/1952] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:11 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:11.208622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:11.223404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f3c9d-7d69-437c-9363-e99690c9bc5a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:11.228915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:11.229906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.230548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.231191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.232779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:11.235550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.254879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 977/1953] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:11.258449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:11.271923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaf1df-300b-4181-808a-236da89a5eb6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:11.285721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:11.286657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.287448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.288154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] PATH_INFO: `/v3/projects/60fbe852023c43379bea111c69b56e43/users/3a293c5cb211499081c4cdd51fbab026/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.313347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fbe852023c43379bea111c69b56e43, user_id=3a293c5cb211499081c4cdd51fbab026, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:11.317024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.336318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 977/1954] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:11 2022] PUT /identity/v3/projects/60fbe852023c43379bea111c69b56e43/users/3a293c5cb211499081c4cdd51fbab026/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:11.340341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:11.352449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c559621-76c0-46bf-ad5f-062052fb5df2 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:11.435483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b957b-66cf-4a45-ad61-213f26cc5436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:11.436102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b957b-66cf-4a45-ad61-213f26cc5436 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.436384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b957b-66cf-4a45-ad61-213f26cc5436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.436667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b957b-66cf-4a45-ad61-213f26cc5436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.437830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b957b-66cf-4a45-ad61-213f26cc5436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:11.439466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b957b-66cf-4a45-ad61-213f26cc5436 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.449904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 978/1955] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:11.455170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:11.465946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498d11a-9803-46e9-8212-d0f9e8b3b848 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:11.471473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:11.472134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.472559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.472843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] PATH_INFO: `/v3/projects/60fbe852023c43379bea111c69b56e43/users/3a293c5cb211499081c4cdd51fbab026/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.474775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fbe852023c43379bea111c69b56e43, user_id=3a293c5cb211499081c4cdd51fbab026, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:11.477599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.488588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 978/1956] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:11 2022] PUT /identity/v3/projects/60fbe852023c43379bea111c69b56e43/users/3a293c5cb211499081c4cdd51fbab026/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:11.498938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.499238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.500047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.520957 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:11.530630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] MFA Rules not processed for user `3a293c5cb211499081c4cdd51fbab026`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:11.532431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:11.618252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 979/1957] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:11.638636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:11.641103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:11.717700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8904bd-3786-46fe-ac46-ac546e30c69e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:11.724175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:11.724901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:11.725231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:11.725540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:11.737031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:11.739035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:11.752870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 979/1958] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:12.582182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:12.598792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c604ec-4671-45b3-bdb4-50e91dce6cbc None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:12.603201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:12.604015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:12.604450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:12.604893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:12.674588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:12.676096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:12.685532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 980/1959] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:19.272706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:19.284949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8f5f-7b2b-46b7-a5b8-e1ff056cba98 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:19.289299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ae4c7-57bd-4d03-bd6c-c5de3b048056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:19.290029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ae4c7-57bd-4d03-bd6c-c5de3b048056 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:19.290384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ae4c7-57bd-4d03-bd6c-c5de3b048056 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:19.290742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ae4c7-57bd-4d03-bd6c-c5de3b048056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:19.346203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:19.370597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ae4c7-57bd-4d03-bd6c-c5de3b048056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:19.372030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ae4c7-57bd-4d03-bd6c-c5de3b048056 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:19.387039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 980/1960] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:19.852841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:19.856941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:19.981992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eca50-33c9-41cb-a0c6-b5fa173cdbfd service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:20.000811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:20.002439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:20.002898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:20.003316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] PATH_INFO: `/v3/users/0d3e8cbc685b4be1acbf2fd0612fa94e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:20.004670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3e8cbc685b4be1acbf2fd0612fa94e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:20.006797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:20.064372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:20.082875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3e8cbc685b4be1acbf2fd0612fa94e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.083793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3e8cbc685b4be1acbf2fd0612fa94e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.093559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3e8cbc685b4be1acbf2fd0612fa94e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.121026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3e8cbc685b4be1acbf2fd0612fa94e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.124270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3e8cbc685b4be1acbf2fd0612fa94e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.152432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 981/1961] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:19 2022] DELETE /identity/v3/users/0d3e8cbc685b4be1acbf2fd0612fa94e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:20.156631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:20.156967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:20.157312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:20.216319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:20.221887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:20.370260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 981/1962] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:20.385275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:20.388647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:20.472779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08879483-c695-4638-860b-90ca9fcc0c0c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:20.476840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:20.477770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:20.477949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:20.478315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:20.488563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:20.490490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:20.502158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 982/1963] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:20.805976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:20.808834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:20.848018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42208d3-0f60-40c3-b1c3-a831c7b5f872 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:20.853916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:20.854759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:20.855152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:20.855429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] PATH_INFO: `/v3/projects/471048704f26423d8b157c0b2effbfe6` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:20.860855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=471048704f26423d8b157c0b2effbfe6)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:20.863650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:20.924054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471048704f26423d8b157c0b2effbfe6', 'request_id': 'req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.931276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471048704f26423d8b157c0b2effbfe6', 'request_id': 'req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.940538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471048704f26423d8b157c0b2effbfe6', 'request_id': 'req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.952174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] The token cache is being invalidate because project 471048704f26423d8b157c0b2effbfe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:20.952576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.953345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:20.954910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 982/1964] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:20 2022] DELETE /identity/v3/projects/471048704f26423d8b157c0b2effbfe6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:20.958916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:20.975812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8323a-72c0-4019-8641-8ecfa01ed482 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:21.042463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:21.043326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:21.043612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:21.043890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] PATH_INFO: `/v3/users/f7f350aaec6443338e7fb01c4b9c0c60` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:21.045273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f350aaec6443338e7fb01c4b9c0c60)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:21.047999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:21.105411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f350aaec6443338e7fb01c4b9c0c60'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.106182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f350aaec6443338e7fb01c4b9c0c60'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.114952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f350aaec6443338e7fb01c4b9c0c60'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.117334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:21.133544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f350aaec6443338e7fb01c4b9c0c60'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.134303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-21f4fdad-ba33-44c4-93a7-e4fc7a217d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f350aaec6443338e7fb01c4b9c0c60'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.148163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 983/1965] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:20 2022] DELETE /identity/v3/users/f7f350aaec6443338e7fb01c4b9c0c60 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:21.453402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:21.455011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:21.478508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67d1d2-ec92-4447-845a-05c4e4fc5ed2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:21.601956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:21.602618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:21.603004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:21.614993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] PATH_INFO: `/v3/projects/11171fc97ffa482fb91550e85271979c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:21.618826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11171fc97ffa482fb91550e85271979c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:21.620960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:21.623060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:21.680518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11171fc97ffa482fb91550e85271979c', 'request_id': 'req-7a326c42-27f7-40fd-b0b6-fe588212d576'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.687503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11171fc97ffa482fb91550e85271979c', 'request_id': 'req-7a326c42-27f7-40fd-b0b6-fe588212d576'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.696495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11171fc97ffa482fb91550e85271979c', 'request_id': 'req-7a326c42-27f7-40fd-b0b6-fe588212d576'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.717612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] The token cache is being invalidate because project 11171fc97ffa482fb91550e85271979c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:21.717964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.719577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7a326c42-27f7-40fd-b0b6-fe588212d576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:21.721383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 983/1966] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:21 2022] DELETE /identity/v3/projects/11171fc97ffa482fb91550e85271979c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:21.727413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:21.727968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:21.728333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:21.751192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:21.752327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:21.820165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 984/1967] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:21.830576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:21.830908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:21.831516 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:21.860355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:21.863024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:21.876694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 984/1968] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:21.881292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:21.892972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded41c-af96-44e3-b810-2eab1d56be03 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:21.901501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:21.902168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:21.902446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:21.903237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:21.903860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:21.904123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:21.906493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:21.920225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 985/1969] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:21.926357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:21.937169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23953e-fdee-4a6c-8ef5-5dcf65be0c12 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:21.942138 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:21.942758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:21.943200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:21.943780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:21.944123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:21.946200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:21.981249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 985/1970] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:21 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:21.985996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:21.998686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982873ac-f38f-44a7-b704-98c25debd334 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.014433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.016343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.064826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.065549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.066449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.066770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.067432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.069395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.091802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab77c0-359b-44b9-982c-2b98d36b881e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.096886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 986/1971] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:21 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:22.097926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.101621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.102328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.102611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.102917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] PATH_INFO: `/v3/users/7882fd175a5a41bbbfa43fdfac7e497f` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.104114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7882fd175a5a41bbbfa43fdfac7e497f)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.106234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.109008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9702-5028-4303-893b-f44ba580096d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.114971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.115929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.116211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.116494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.117138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.118724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.131765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 987/1972] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:22.135200 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.151120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d4fd3-952b-4695-8ef0-16d957858484 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.156772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.157385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.157661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.157939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.158584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.160202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.161517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7882fd175a5a41bbbfa43fdfac7e497f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.162193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7882fd175a5a41bbbfa43fdfac7e497f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.162853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7882fd175a5a41bbbfa43fdfac7e497f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.173801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 988/1973] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:22.174033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7882fd175a5a41bbbfa43fdfac7e497f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.175781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.177585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.179591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:22.190097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-57cfba42-a986-45ad-9f1d-115613cccd52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7882fd175a5a41bbbfa43fdfac7e497f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.191122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ae22-8f25-49f9-a153-d6a0e4626003 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.195912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4f509-6ba5-48f3-9cf5-9c30570da023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.196604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4f509-6ba5-48f3-9cf5-9c30570da023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.196912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4f509-6ba5-48f3-9cf5-9c30570da023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.197191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4f509-6ba5-48f3-9cf5-9c30570da023 admin admin] PATH_INFO: `/v3/projects/9a47be381ddb4668afc2f81210b54ea4/users/488dbdf91e764b019aa1207cf8e67bb7/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.205199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 986/1974] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:22 2022] DELETE /identity/v3/users/7882fd175a5a41bbbfa43fdfac7e497f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:22.209982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.210291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.210566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.215564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4f509-6ba5-48f3-9cf5-9c30570da023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a47be381ddb4668afc2f81210b54ea4, user_id=488dbdf91e764b019aa1207cf8e67bb7, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.219056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4f509-6ba5-48f3-9cf5-9c30570da023 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.232406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:22.233640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.234796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 989/1975] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:22 2022] PUT /identity/v3/projects/9a47be381ddb4668afc2f81210b54ea4/users/488dbdf91e764b019aa1207cf8e67bb7/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:22.245344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.245652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.246000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.259822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:22.277233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] MFA Rules not processed for user `488dbdf91e764b019aa1207cf8e67bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:22.278463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.299250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 987/1976] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:22.311475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.313423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.348986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 990/1977] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:22.353349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.366348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a12da0-a444-4fb4-ab1a-8aec902e9ff0 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.372310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.373035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.373916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.374443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.376913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.379046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.386666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d2531-95de-41c8-b3ee-96679e415dfc None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.402568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.403277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.403736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.404074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.414046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 991/1978] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:22 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:22.414222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.415695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.417868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.426405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 988/1979] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:22.493522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a1fc3-5ca9-4e34-b190-4cacaeb525f9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.499174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.500617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.501164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.501532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.502508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.504879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.527945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 992/1980] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:22 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:22.532488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.542157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070897e4-3fb0-48d0-b667-7c9f51551e97 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.546827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.547446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.547722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.547988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.548649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.550326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.566174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 989/1981] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:22.570455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.582618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c180c-3793-4dd4-9cf1-f5ca92a06aa5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.588114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.588949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.589348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.589786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] PATH_INFO: `/v3/projects/774821e32b634850bd793b221ba5876a/users/fca29545f9264a8fbbe63b26f3a60569/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.607030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774821e32b634850bd793b221ba5876a, user_id=fca29545f9264a8fbbe63b26f3a60569, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.609852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.624032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 993/1982] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:22 2022] PUT /identity/v3/projects/774821e32b634850bd793b221ba5876a/users/fca29545f9264a8fbbe63b26f3a60569/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:22.629139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.641145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964732c-2edb-446a-84b9-1fa1934792df admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.706644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654cf78-9f31-4e45-b311-75732fdf1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.707286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654cf78-9f31-4e45-b311-75732fdf1bae admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.707568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654cf78-9f31-4e45-b311-75732fdf1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.707844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654cf78-9f31-4e45-b311-75732fdf1bae admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.708483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654cf78-9f31-4e45-b311-75732fdf1bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.710080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654cf78-9f31-4e45-b311-75732fdf1bae admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.723807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 990/1983] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:22.725145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.737396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d2cbd-b57c-462b-9df5-efc1f3f9a590 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.749876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.750786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.751183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.751538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] PATH_INFO: `/v3/projects/774821e32b634850bd793b221ba5876a/users/fca29545f9264a8fbbe63b26f3a60569/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.754085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774821e32b634850bd793b221ba5876a, user_id=fca29545f9264a8fbbe63b26f3a60569, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.757185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.771470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 994/1984] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:22 2022] PUT /identity/v3/projects/774821e32b634850bd793b221ba5876a/users/fca29545f9264a8fbbe63b26f3a60569/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:22.782970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.783360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.783719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.784672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.786185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.805214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:22.816689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] MFA Rules not processed for user `fca29545f9264a8fbbe63b26f3a60569`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:22.819577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.845182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:22.886617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e785-04ed-4feb-b1de-07122bbb7434 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:22.891886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:22.892578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:22.892864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:22.893136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] PATH_INFO: `/v3/projects/0eae974749644160b8e94e9efe8026b0` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:22.894255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eae974749644160b8e94e9efe8026b0)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:22.896196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:22.907750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 991/1985] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:22.918982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.920847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.947730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eae974749644160b8e94e9efe8026b0', 'request_id': 'req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.953566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eae974749644160b8e94e9efe8026b0', 'request_id': 'req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.961704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eae974749644160b8e94e9efe8026b0', 'request_id': 'req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.985644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] The token cache is being invalidate because project 0eae974749644160b8e94e9efe8026b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:22.985880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.986532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:22.991378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 995/1986] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:22 2022] DELETE /identity/v3/projects/0eae974749644160b8e94e9efe8026b0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:22.992233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:22.993569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:22.995489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce944e-5be8-459c-afaf-1194f81b2a7d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:23.061884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:23.063303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:23.063654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:23.064051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:23.064522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e426-eba5-4a82-9cfc-f7d1c8c62fc7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:23.065951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:23.069558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:23.069933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:23.070219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:23.070500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] PATH_INFO: `/v3/users/80cbac74fb874b3f816a354fad776632` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:23.071693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80cbac74fb874b3f816a354fad776632)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:23.074023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:23.123722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80cbac74fb874b3f816a354fad776632'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.125018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80cbac74fb874b3f816a354fad776632'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.132336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:23.132680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80cbac74fb874b3f816a354fad776632'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.133820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:23.134554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:23.140295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:23.146659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 992/1987] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:00:23.148372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80cbac74fb874b3f816a354fad776632'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.149090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80cbac74fb874b3f816a354fad776632'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.165205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 996/1988] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:22 2022] DELETE /identity/v3/users/80cbac74fb874b3f816a354fad776632 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:23.198606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:23.200078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:23.212366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0227b1-c689-4ad6-a519-d924bfd0dd23 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:23.269453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:23.270207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:23.270389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:23.270770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:23.272634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:23.342092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:23.344517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:23.355033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 993/1989] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:23.463161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:23.463161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:23.549569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d4b43-040c-4349-92bd-a65064cea72b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:23.554418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:23.555080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:23.555363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:23.555633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] PATH_INFO: `/v3/projects/da5b167f189c4aa1b8f89805681a626b` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:23.556955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5b167f189c4aa1b8f89805681a626b)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:23.558896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:23.611812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5b167f189c4aa1b8f89805681a626b', 'request_id': 'req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.618066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5b167f189c4aa1b8f89805681a626b', 'request_id': 'req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.625420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5b167f189c4aa1b8f89805681a626b', 'request_id': 'req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.639079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] The token cache is being invalidate because project da5b167f189c4aa1b8f89805681a626b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:23.639379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.645559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.647349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 997/1990] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:23 2022] DELETE /identity/v3/projects/da5b167f189c4aa1b8f89805681a626b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:23.651782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:23.667609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54015229-d27c-4ae8-89af-cd92f3484ff9 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:23.744756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:23.745429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:23.745710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:23.746010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] PATH_INFO: `/v3/users/3a293c5cb211499081c4cdd51fbab026` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:23.747158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a293c5cb211499081c4cdd51fbab026)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:23.749094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:23.795783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a293c5cb211499081c4cdd51fbab026'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.797150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a293c5cb211499081c4cdd51fbab026'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.797997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a293c5cb211499081c4cdd51fbab026'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.804849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a293c5cb211499081c4cdd51fbab026'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.805944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:23.819888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-47834db7-e950-4e8e-80bd-812ff77b8dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a293c5cb211499081c4cdd51fbab026'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:23.834060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 994/1991] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:23 2022] DELETE /identity/v3/users/3a293c5cb211499081c4cdd51fbab026 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:24.212262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.213818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:24.233689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6737dfb-1e1f-4310-85f5-4a675ee53ae6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.295396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.296036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.297007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.297270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] PATH_INFO: `/v3/projects/60fbe852023c43379bea111c69b56e43` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.298426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60fbe852023c43379bea111c69b56e43)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.300356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.308544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:24.352188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60fbe852023c43379bea111c69b56e43', 'request_id': 'req-28396bab-ea3a-4609-a53d-320cab8120b1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:24.357672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60fbe852023c43379bea111c69b56e43', 'request_id': 'req-28396bab-ea3a-4609-a53d-320cab8120b1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:24.363986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60fbe852023c43379bea111c69b56e43', 'request_id': 'req-28396bab-ea3a-4609-a53d-320cab8120b1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:24.377149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] The token cache is being invalidate because project 60fbe852023c43379bea111c69b56e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:24.377440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:24.378074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-28396bab-ea3a-4609-a53d-320cab8120b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:24.379621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 998/1992] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:24 2022] DELETE /identity/v3/projects/60fbe852023c43379bea111c69b56e43 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:24.387741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.388121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.388498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.409613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:24.410830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:24.471728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 995/1993] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:24.480672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.481003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.481235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.501055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:24.502269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:24.510379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 999/1994] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:24.514694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.525046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed00eff-7511-4da0-9bd8-d1c8cdcce032 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.529641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.530236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.530514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.531435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.532069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:24.532368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.533958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.545223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 996/1995] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:24.549875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.560453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2bf07-0df4-40b0-ac2c-79e282ddac0c None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.565124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.565733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.566014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.566285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.567051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.569371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.598575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1000/1996] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:24 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:24.602762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.613286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02bcd2-6262-4ed0-b430-4bfdbf455083 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.671927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.672602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.672798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.673069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.673708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.675653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.700766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 997/1997] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:24 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:24.701948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.713914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c7b51-85cd-47b7-8aca-15ed9a57ac5b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.719146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.719794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.720058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.720359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.721012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.722633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.733009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1001/1998] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:24.737115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.747201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce9b9-c15f-4e8e-86a5-d913be0d01c3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.753080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0474a-c3c1-4ec6-9b84-f4dd5efb73c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.753682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0474a-c3c1-4ec6-9b84-f4dd5efb73c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.753965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0474a-c3c1-4ec6-9b84-f4dd5efb73c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.754236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0474a-c3c1-4ec6-9b84-f4dd5efb73c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.754887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0474a-c3c1-4ec6-9b84-f4dd5efb73c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.756467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0474a-c3c1-4ec6-9b84-f4dd5efb73c1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.779505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 998/1999] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:24.780810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:24.790984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42e539-7ec5-426d-953b-fe789656420d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:24.796026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:24.797110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.797389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.797661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] PATH_INFO: `/v3/projects/abd919bcd86e4bcfb79b7c86f0732b3f/users/9275dd82b8db4102add8d284dc60add2/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.813258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd919bcd86e4bcfb79b7c86f0732b3f, user_id=9275dd82b8db4102add8d284dc60add2, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:24.816088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:24.830340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1002/2000] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:24 2022] PUT /identity/v3/projects/abd919bcd86e4bcfb79b7c86f0732b3f/users/9275dd82b8db4102add8d284dc60add2/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:24.841510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:24.841510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:24.841769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:24.872029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] MFA Rules not processed for user `9275dd82b8db4102add8d284dc60add2`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:24.873254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:24.909482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:24.949262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 999/2001] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:24.954403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.019287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767811dd-6d1a-4f93-a190-255c5357c825 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.024420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.025035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.025314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.025615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.026227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.028170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.061599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1003/2002] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:24 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:25.066395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.084015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bcf4a-2f4c-42ee-9252-018c8325ba89 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.170017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.170904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.172172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.172644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.173460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.175744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.207204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1000/2003] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:25 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:25.208134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.220386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02199efc-dd18-4a57-97a5-1e854f890e3e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.231072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.231072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.231411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.231806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.232761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.235264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.258346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1004/2004] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:25.261229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.273620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9114ae2-b63e-44aa-be7f-9aca319738b2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.279048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c15dfd-29ba-4122-859f-ab74c37ab268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.279857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c15dfd-29ba-4122-859f-ab74c37ab268 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.280192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c15dfd-29ba-4122-859f-ab74c37ab268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.280901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c15dfd-29ba-4122-859f-ab74c37ab268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.281870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c15dfd-29ba-4122-859f-ab74c37ab268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.284087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c15dfd-29ba-4122-859f-ab74c37ab268 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.294870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1001/2005] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:25.299831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.309897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b171eb-3503-4be5-8a90-400f5fe1406c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.315672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.316268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.316554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.316856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] PATH_INFO: `/v3/projects/cc36ab320af6443c96d6dcf05e29b896/users/8fd0b75ac9cd430199848f2590f5912c/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.325301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:25.333690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc36ab320af6443c96d6dcf05e29b896, user_id=8fd0b75ac9cd430199848f2590f5912c, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.336537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.349737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1005/2006] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:25 2022] PUT /identity/v3/projects/cc36ab320af6443c96d6dcf05e29b896/users/8fd0b75ac9cd430199848f2590f5912c/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:25.361348 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.361635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.361914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.390200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] MFA Rules not processed for user `8fd0b75ac9cd430199848f2590f5912c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:25.391491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:25.455635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1002/2007] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:25.460078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.522385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a7fc7-97cf-4daf-946d-98b9321358e4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.527107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.527716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.527998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.528274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.528920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.531515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.565814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1006/2008] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:25 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:25.566893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.578520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dafd45-4642-4175-b282-1c9c032c7a1c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.627284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:25.653357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.653984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.654402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.654686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.655374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.658104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.681338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1003/2009] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:25 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:25.685652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.696019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4147c-f693-449c-87e9-e3c2e0f246af admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.700882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.701479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.701759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.702031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.702736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.704355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.715026 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1007/2010] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:25.719708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.729765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cf4f6-9638-4105-8490-8faf7d72d5b7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.735031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.735758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.736154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.736427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] PATH_INFO: `/v3/projects/a707e3afa4a0404db374ceb09d7248be/users/9cdacf463b424ffdb51c6f64dddfde2d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.752350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a707e3afa4a0404db374ceb09d7248be, user_id=9cdacf463b424ffdb51c6f64dddfde2d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.755045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.768534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1004/2011] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:25 2022] PUT /identity/v3/projects/a707e3afa4a0404db374ceb09d7248be/users/9cdacf463b424ffdb51c6f64dddfde2d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:25.772699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.783214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80e180-5e06-4aca-a3fa-400572e1a407 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.844159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46cbdd-7e53-4260-a79b-4215d94d49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.844685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46cbdd-7e53-4260-a79b-4215d94d49be admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.844964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46cbdd-7e53-4260-a79b-4215d94d49be admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.845236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46cbdd-7e53-4260-a79b-4215d94d49be admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.845869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46cbdd-7e53-4260-a79b-4215d94d49be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.847481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46cbdd-7e53-4260-a79b-4215d94d49be admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.857050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1008/2012] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:25.861665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:25.871774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5aceb-9689-4e98-a42a-0f2f28f65470 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:25.877045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:25.877706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.877988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.878263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] PATH_INFO: `/v3/projects/a707e3afa4a0404db374ceb09d7248be/users/9cdacf463b424ffdb51c6f64dddfde2d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.880268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a707e3afa4a0404db374ceb09d7248be, user_id=9cdacf463b424ffdb51c6f64dddfde2d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:25.883032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:25.895812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1005/2013] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:25 2022] PUT /identity/v3/projects/a707e3afa4a0404db374ceb09d7248be/users/9cdacf463b424ffdb51c6f64dddfde2d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:25.907189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:25.907482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:25.907765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:25.935393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] MFA Rules not processed for user `9cdacf463b424ffdb51c6f64dddfde2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:25.936643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:25.984213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:26.014704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1009/2014] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:26.027422 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:26.029174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:26.087929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6544-93dd-4201-a746-ea628198655c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:26.091642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ce93-3d3e-402f-9830-1a22c7949a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:26.092254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ce93-3d3e-402f-9830-1a22c7949a4d service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:26.092540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ce93-3d3e-402f-9830-1a22c7949a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:26.092815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ce93-3d3e-402f-9830-1a22c7949a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:26.154050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ce93-3d3e-402f-9830-1a22c7949a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:26.156141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ce93-3d3e-402f-9830-1a22c7949a4d service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:26.166194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1006/2015] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:26.923437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:26.925105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:26.995508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38793d2-f3a3-4f03-8f3b-8f72771318a0 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:27.000505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:27.001143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:27.001426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:27.001704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] PATH_INFO: `/v3/users/488dbdf91e764b019aa1207cf8e67bb7` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:27.002952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=488dbdf91e764b019aa1207cf8e67bb7)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:27.004891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:27.054892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '488dbdf91e764b019aa1207cf8e67bb7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.055550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '488dbdf91e764b019aa1207cf8e67bb7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.062960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '488dbdf91e764b019aa1207cf8e67bb7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.075131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '488dbdf91e764b019aa1207cf8e67bb7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.075825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '488dbdf91e764b019aa1207cf8e67bb7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.089874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1010/2016] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:26 2022] DELETE /identity/v3/users/488dbdf91e764b019aa1207cf8e67bb7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:27.095492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:27.095997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:27.096088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:27.116370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:27.118004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:27.159967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:27.189306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1007/2017] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:27.209422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:27.211486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:27.292132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c42c8b-2435-4538-aa0e-c21ca3f13287 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:27.295953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:27.297233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:27.297540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:27.297825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:27.310747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:27.312579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:27.319020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:27.325992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1011/2018] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:27.676411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:27.678175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:27.710089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264b105-2d02-4c02-af7f-6cd80dd89657 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:27.721981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:27.722801 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:27.723126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:27.723405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] PATH_INFO: `/v3/projects/9a47be381ddb4668afc2f81210b54ea4` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:27.724564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a47be381ddb4668afc2f81210b54ea4)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:27.727539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:27.802674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:27.804340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:27.823894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a47be381ddb4668afc2f81210b54ea4', 'request_id': 'req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.832814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a47be381ddb4668afc2f81210b54ea4', 'request_id': 'req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.845244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a47be381ddb4668afc2f81210b54ea4', 'request_id': 'req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.860391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] The token cache is being invalidate because project 9a47be381ddb4668afc2f81210b54ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:27.860752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.861410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:27.863364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1008/2019] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:27 2022] DELETE /identity/v3/projects/9a47be381ddb4668afc2f81210b54ea4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:27.867122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:27.868630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:27.893704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d795990-b375-4221-8059-7864ca07b9de service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:27.959634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:27.960653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:27.960653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:27.960999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:27.962572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:27.964519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b43a16-f38b-40ee-8a0c-3ca38e0567cf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:27.971599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:27.972510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:27.973700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:27.973979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] PATH_INFO: `/v3/users/fca29545f9264a8fbbe63b26f3a60569` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:27.975212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca29545f9264a8fbbe63b26f3a60569)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:27.981309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:28.048062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca29545f9264a8fbbe63b26f3a60569'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.049148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca29545f9264a8fbbe63b26f3a60569'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.049831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca29545f9264a8fbbe63b26f3a60569'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.053129 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:28.054582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:28.057770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca29545f9264a8fbbe63b26f3a60569'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.066310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1012/2020] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:00:28.070048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:28.072653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-5b4307dc-6262-4ae4-8958-e03ebf4071d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca29545f9264a8fbbe63b26f3a60569'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.097304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1009/2021] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:27 2022] DELETE /identity/v3/users/fca29545f9264a8fbbe63b26f3a60569 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:28.461122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:28.473078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26deeae2-511f-483b-85bc-450eec7436b2 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:28.536694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:28.537382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:28.537692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:28.537966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] PATH_INFO: `/v3/projects/774821e32b634850bd793b221ba5876a` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:28.539166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] RBAC: Authorizing `identity:delete_project(project_id=774821e32b634850bd793b221ba5876a)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:28.541158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:28.553856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:28.594530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774821e32b634850bd793b221ba5876a', 'request_id': 'req-db36c0b5-09e8-4858-9648-ab37f36742db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.600991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774821e32b634850bd793b221ba5876a', 'request_id': 'req-db36c0b5-09e8-4858-9648-ab37f36742db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.607571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774821e32b634850bd793b221ba5876a', 'request_id': 'req-db36c0b5-09e8-4858-9648-ab37f36742db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.620181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] The token cache is being invalidate because project 774821e32b634850bd793b221ba5876a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:28.620476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.621130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-db36c0b5-09e8-4858-9648-ab37f36742db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:28.622671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1013/2022] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:28 2022] DELETE /identity/v3/projects/774821e32b634850bd793b221ba5876a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:28.628560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:28.628921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:28.629304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:28.650125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:28.651354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:28.715056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1010/2023] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:28.724682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:28.725039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:28.725391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:28.752335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:28.753538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:28.762262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1014/2024] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:28.768267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:28.782234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82edf3-5946-4694-877d-eea4f8b56e83 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:28.782916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:28.784536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:28.808886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:28.809553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:28.809845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:28.810121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:28.810849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:28.811194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:28.813458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:28.827010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1011/2025] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:28.832078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:28.843077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb0484-fde5-4250-92be-a06529cd97fd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:28.854320 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:28.855334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:28.855804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:28.856405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:28.857364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:28.860722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:28.865977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842ffb5-3f9e-45c4-88f9-bbf7c2702081 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:28.871539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:28.872467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:28.872893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:28.873174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:28.876130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:28.908344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1012/2026] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:28 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:28.915977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:28.969786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:28.976343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.001746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c50dcf-ed88-40dd-9992-ea6da3cd91e8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.014412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1015/2027] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:00:29.021540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.022237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.022997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.023411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.024451 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.026568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.057442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1013/2028] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:28 2022] POST /identity/v3/users => generated 301 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:29.061606 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.082359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954347-8bfa-418e-8d8f-6c4b31739eb4 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.088979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.089607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.089893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.090167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.090881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.092522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.104620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1016/2029] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:29.109260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.119748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de96265-f96b-4152-938b-1fda4a13a152 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.126973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99695f55-eb63-4e9c-af93-fb4255fe8c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.127954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99695f55-eb63-4e9c-af93-fb4255fe8c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.128392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99695f55-eb63-4e9c-af93-fb4255fe8c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.128832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99695f55-eb63-4e9c-af93-fb4255fe8c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.129481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99695f55-eb63-4e9c-af93-fb4255fe8c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.131762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99695f55-eb63-4e9c-af93-fb4255fe8c3e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.142203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1014/2030] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:29.145997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.157490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ce1a5-1bc2-438a-834a-907ce9aa5d3e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.162484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.163185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.163471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.163749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] PATH_INFO: `/v3/projects/ed52907582e840a3abe1639382b44a4a/users/7705c5f6b7344047812ad90356c02ed3/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.179864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed52907582e840a3abe1639382b44a4a, user_id=7705c5f6b7344047812ad90356c02ed3, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.182562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.196676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1017/2031] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:29 2022] PUT /identity/v3/projects/ed52907582e840a3abe1639382b44a4a/users/7705c5f6b7344047812ad90356c02ed3/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:29.207623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.207917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.208198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.239955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] MFA Rules not processed for user `7705c5f6b7344047812ad90356c02ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:29.241088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:29.260023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:29.326515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1015/2032] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:29 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:29.331302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.342271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:29.414408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7321a-5fca-4512-8a1f-b3dd9c814bb8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.419104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.419710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.420099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.420357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.420997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.423628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.453500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1018/2033] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:29 2022] POST /identity/v3/projects => generated 401 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:29.454944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.465671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea6e79-1e22-4c02-a8c1-fb1d19c49086 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.522127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.522655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.522959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.523231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.523861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.525783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.548148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1016/2034] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:29 2022] POST /identity/v3/users => generated 307 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:29.552474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.563918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c8293-9493-4024-99ef-9a3131ef3f82 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.578353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.579293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.579580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.579892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.580552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.582157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.593513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1019/2035] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:29.598991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.609553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993084-54bc-4b01-a4d2-f0bcaaa34726 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.614131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.614786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.615104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.615381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] PATH_INFO: `/v3/projects/4e094a4a575b47b6a95cde5d386b0a9c/users/1c1fbfad5db34e0eadc382214575d8fe/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.630396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e094a4a575b47b6a95cde5d386b0a9c, user_id=1c1fbfad5db34e0eadc382214575d8fe, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.633093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.648121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1017/2036] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:29 2022] PUT /identity/v3/projects/4e094a4a575b47b6a95cde5d386b0a9c/users/1c1fbfad5db34e0eadc382214575d8fe/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:29.652415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.667816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5bc32-9484-4bc1-9c15-176416fa569d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.733998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9207a47-687d-48ad-a291-529e4d19217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.734815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9207a47-687d-48ad-a291-529e4d19217f admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.735216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9207a47-687d-48ad-a291-529e4d19217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.735673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9207a47-687d-48ad-a291-529e4d19217f admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.736330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9207a47-687d-48ad-a291-529e4d19217f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.738568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9207a47-687d-48ad-a291-529e4d19217f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.752716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1020/2037] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:29.757405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.769093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042c58a-8b09-45b6-9f43-0bc11356598a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:29.774150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:29.774722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.775029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.775301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] PATH_INFO: `/v3/projects/4e094a4a575b47b6a95cde5d386b0a9c/users/1c1fbfad5db34e0eadc382214575d8fe/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.777277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e094a4a575b47b6a95cde5d386b0a9c, user_id=1c1fbfad5db34e0eadc382214575d8fe, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:29.785921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:29.797522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1018/2038] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:29 2022] PUT /identity/v3/projects/4e094a4a575b47b6a95cde5d386b0a9c/users/1c1fbfad5db34e0eadc382214575d8fe/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:29.808792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:29.809088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:29.809365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:29.847161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] MFA Rules not processed for user `1c1fbfad5db34e0eadc382214575d8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:29.848396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:29.926889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1021/2039] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:29 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:29.941834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:29.944069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:30.023740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04197d-4da8-4ac5-a3f7-fda9149cd784 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:30.027406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:30.028018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:30.028296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:30.028566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:30.038467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:30.039941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:30.052102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1019/2040] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:29 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:30.145617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:30.161525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:30.172554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3d070-6fa5-4bc7-8f19-92ffe87c6753 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:30.182991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce0d9d-4de3-4ebb-aa81-57a400cd734a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:30.183679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce0d9d-4de3-4ebb-aa81-57a400cd734a service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:30.183974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce0d9d-4de3-4ebb-aa81-57a400cd734a service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:30.184254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce0d9d-4de3-4ebb-aa81-57a400cd734a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:30.251124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce0d9d-4de3-4ebb-aa81-57a400cd734a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:30.252567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce0d9d-4de3-4ebb-aa81-57a400cd734a service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:30.265758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1022/2041] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:30 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:33.242698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:33.244661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:33.285573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:33.362038 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6283-335d-48b9-a00f-ea29e031bf19 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:33.369158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:33.369903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:33.370340 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:33.370685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] PATH_INFO: `/v3/users/9275dd82b8db4102add8d284dc60add2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:33.373182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9275dd82b8db4102add8d284dc60add2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:33.376226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:33.473755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9275dd82b8db4102add8d284dc60add2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:33.474534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9275dd82b8db4102add8d284dc60add2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:33.476091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9275dd82b8db4102add8d284dc60add2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:33.486545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9275dd82b8db4102add8d284dc60add2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:33.519391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9275dd82b8db4102add8d284dc60add2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:33.534683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1020/2042] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:33 2022] DELETE /identity/v3/users/9275dd82b8db4102add8d284dc60add2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:33.539703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:33.540234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:33.540234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:33.561602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:33.563234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:33.636333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1023/2043] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:33.648084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:33.650115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:33.712585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f03e8-f948-41c0-bbee-777511e52f00 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:33.718277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:33.720062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:33.720507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:33.720956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:33.732376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:33.734287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:33.746854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1021/2044] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:34.021270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:34.022884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:34.043440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a572122-00e7-4d9d-a00d-d113face85e7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:34.048616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:34.049600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:34.049897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:34.050217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] PATH_INFO: `/v3/projects/abd919bcd86e4bcfb79b7c86f0732b3f` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:34.051475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd919bcd86e4bcfb79b7c86f0732b3f)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:34.053464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:34.118671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd919bcd86e4bcfb79b7c86f0732b3f', 'request_id': 'req-b7f97352-eebe-4216-ab8e-b1db8112fed5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.126287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd919bcd86e4bcfb79b7c86f0732b3f', 'request_id': 'req-b7f97352-eebe-4216-ab8e-b1db8112fed5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.132551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd919bcd86e4bcfb79b7c86f0732b3f', 'request_id': 'req-b7f97352-eebe-4216-ab8e-b1db8112fed5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.144961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] The token cache is being invalidate because project abd919bcd86e4bcfb79b7c86f0732b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:34.145382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.146090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.147634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1024/2045] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:34 2022] DELETE /identity/v3/projects/abd919bcd86e4bcfb79b7c86f0732b3f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:34.151879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:34.162387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f43b49-6feb-401a-af80-9223073c3959 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:34.201306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:34.226171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:34.226767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:34.227078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:34.227358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] PATH_INFO: `/v3/users/8fd0b75ac9cd430199848f2590f5912c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:34.228503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd0b75ac9cd430199848f2590f5912c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:34.230503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:34.281461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd0b75ac9cd430199848f2590f5912c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.282266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd0b75ac9cd430199848f2590f5912c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.282972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd0b75ac9cd430199848f2590f5912c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.291138 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd0b75ac9cd430199848f2590f5912c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.305336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd0b75ac9cd430199848f2590f5912c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.318912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1022/2046] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:34 2022] DELETE /identity/v3/users/8fd0b75ac9cd430199848f2590f5912c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:34.602916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:34.604493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:34.622468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f97352-eebe-4216-ab8e-b1db8112fed5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:34.653851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:34.695960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:34.696583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:34.697010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:34.697225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] PATH_INFO: `/v3/projects/cc36ab320af6443c96d6dcf05e29b896` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:34.701579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc36ab320af6443c96d6dcf05e29b896)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:34.701579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:34.721262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:34.722809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:34.738006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfc40d-c08e-4e2c-8839-41c1a7f86623 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:34.750397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc36ab320af6443c96d6dcf05e29b896', 'request_id': 'req-2ca54748-6cb8-4bf9-b996-37f5c8396968'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.756678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc36ab320af6443c96d6dcf05e29b896', 'request_id': 'req-2ca54748-6cb8-4bf9-b996-37f5c8396968'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.763324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc36ab320af6443c96d6dcf05e29b896', 'request_id': 'req-2ca54748-6cb8-4bf9-b996-37f5c8396968'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.778228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] The token cache is being invalidate because project cc36ab320af6443c96d6dcf05e29b896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:34.778524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.779186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.781653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1025/2047] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:34 2022] DELETE /identity/v3/projects/cc36ab320af6443c96d6dcf05e29b896 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:34.785349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:34.786738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:34.800535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:34.801182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:34.801464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:34.802389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] PATH_INFO: `/v3/users/7705c5f6b7344047812ad90356c02ed3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:34.804228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7705c5f6b7344047812ad90356c02ed3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:34.806195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:34.867044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca54748-6cb8-4bf9-b996-37f5c8396968 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:34.867149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7705c5f6b7344047812ad90356c02ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.867149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7705c5f6b7344047812ad90356c02ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.867149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7705c5f6b7344047812ad90356c02ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.872195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:34.872195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:34.872195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:34.872195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] PATH_INFO: `/v3/users/9cdacf463b424ffdb51c6f64dddfde2d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:34.873305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cdacf463b424ffdb51c6f64dddfde2d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:34.874224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7705c5f6b7344047812ad90356c02ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.876088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:34.879675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:34.895958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-131796ce-6b7d-42c9-b359-28f272b7abd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7705c5f6b7344047812ad90356c02ed3'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.914995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1023/2048] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:34 2022] DELETE /identity/v3/users/7705c5f6b7344047812ad90356c02ed3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:34.920242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:34.921938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:34.922408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:34.943969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cdacf463b424ffdb51c6f64dddfde2d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.945332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cdacf463b424ffdb51c6f64dddfde2d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.947843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:34.949408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:34.959531 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cdacf463b424ffdb51c6f64dddfde2d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.962435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:34.978703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cdacf463b424ffdb51c6f64dddfde2d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.980144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cdacf463b424ffdb51c6f64dddfde2d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:34.999980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1026/2049] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:34 2022] DELETE /identity/v3/users/9cdacf463b424ffdb51c6f64dddfde2d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:35.103096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1024/2050] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:35.113936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.115970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.192634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b287d-dc0e-44e5-a786-c58e95cfb13b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.197132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9bd9d-0470-4dfc-8e35-03dd816576ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.197779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9bd9d-0470-4dfc-8e35-03dd816576ef service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.198516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9bd9d-0470-4dfc-8e35-03dd816576ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.199081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9bd9d-0470-4dfc-8e35-03dd816576ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.295524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9bd9d-0470-4dfc-8e35-03dd816576ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.295524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9bd9d-0470-4dfc-8e35-03dd816576ef service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.306979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1027/2051] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:35.451115 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.453139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.465163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b614cd-9a17-46a7-964e-fdd164726a60 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.471155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.471995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.472283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.472556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] PATH_INFO: `/v3/projects/a707e3afa4a0404db374ceb09d7248be` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.473737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a707e3afa4a0404db374ceb09d7248be)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.475887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.527823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a707e3afa4a0404db374ceb09d7248be', 'request_id': 'req-067a09c2-368c-4f5e-a05f-e0e3589653c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.536437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a707e3afa4a0404db374ceb09d7248be', 'request_id': 'req-067a09c2-368c-4f5e-a05f-e0e3589653c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.545152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a707e3afa4a0404db374ceb09d7248be', 'request_id': 'req-067a09c2-368c-4f5e-a05f-e0e3589653c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.556702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] The token cache is being invalidate because project a707e3afa4a0404db374ceb09d7248be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:35.556986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.557660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-067a09c2-368c-4f5e-a05f-e0e3589653c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.559192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1025/2052] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:35 2022] DELETE /identity/v3/projects/a707e3afa4a0404db374ceb09d7248be => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:35.565068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.565385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.565660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.587157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:35.589682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.655315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1028/2053] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:35.664775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.665358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.665644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.666098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.667337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.680789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504e341-0738-493b-9c30-f67e29e96645 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.685574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.686296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.686583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.687010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] PATH_INFO: `/v3/projects/ed52907582e840a3abe1639382b44a4a` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.688665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed52907582e840a3abe1639382b44a4a)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.690672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.694034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:35.695286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.709141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1026/2054] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:35.710368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.716902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:35.721654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30724a-47ae-4531-91a0-c71d2174ca25 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.727567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.728172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.728447 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.728718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.729359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:35.729661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.731329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.743948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed52907582e840a3abe1639382b44a4a', 'request_id': 'req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.746885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1027/2055] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:35.750396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.753344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed52907582e840a3abe1639382b44a4a', 'request_id': 'req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.759552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed52907582e840a3abe1639382b44a4a', 'request_id': 'req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.760583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a9017-7c09-4506-8df8-93d30db74596 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.773140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] The token cache is being invalidate because project ed52907582e840a3abe1639382b44a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:35.773465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.774107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.775778 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1029/2056] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:35 2022] DELETE /identity/v3/projects/ed52907582e840a3abe1639382b44a4a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:35.780416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.781541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.800699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:35.824381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.825412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.825702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.825975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.828033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.829988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.854215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56a0ac-5b3a-4028-b2b8-ad02d0318482 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.859868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.860580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.860938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.861287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] PATH_INFO: `/v3/users/1c1fbfad5db34e0eadc382214575d8fe` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.862758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1fbfad5db34e0eadc382214575d8fe)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.868087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1028/2057] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:35 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:35.868247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.869256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.871007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:35.933668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1fbfad5db34e0eadc382214575d8fe'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.934615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1fbfad5db34e0eadc382214575d8fe'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.943314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1fbfad5db34e0eadc382214575d8fe'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.950434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50ce8b-7da9-4848-9105-32345655afc0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:35.955987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:35.956700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:35.956981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:35.957088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1fbfad5db34e0eadc382214575d8fe'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.957251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:35.957852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1fbfad5db34e0eadc382214575d8fe'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:35.957964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:35.959901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:35.974293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1030/2058] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:35 2022] DELETE /identity/v3/users/1c1fbfad5db34e0eadc382214575d8fe => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:35.990712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1029/2059] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:35 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:35.994775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:35.996278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.030124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:36.086416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b32885-de18-47c0-8ec8-074efa2fecd5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.096169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.097018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.097657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.098020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.098898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.100708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.113970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1031/2060] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:36.118585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.133623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89091c5-ba25-4d7e-b44d-884da1c80508 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.144491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054db48-4b5f-4867-8eb1-020dbfa4effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.145214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054db48-4b5f-4867-8eb1-020dbfa4effe admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.145549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054db48-4b5f-4867-8eb1-020dbfa4effe admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.145912 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054db48-4b5f-4867-8eb1-020dbfa4effe admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.146620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054db48-4b5f-4867-8eb1-020dbfa4effe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.148511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054db48-4b5f-4867-8eb1-020dbfa4effe admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.170978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1030/2061] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:36.174222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.190976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f79a8c-35c4-4ab2-99c2-2a122153eceb admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.197706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.198791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.199304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.199711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] PATH_INFO: `/v3/projects/b69abb36de2141628e74868c582e079d/users/11be0eea7b634533a9b72dfd5d76b9a7/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.223496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69abb36de2141628e74868c582e079d, user_id=11be0eea7b634533a9b72dfd5d76b9a7, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.226819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.244820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1032/2062] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:36 2022] PUT /identity/v3/projects/b69abb36de2141628e74868c582e079d/users/11be0eea7b634533a9b72dfd5d76b9a7/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:36.255943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.256442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.256623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.291294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] MFA Rules not processed for user `11be0eea7b634533a9b72dfd5d76b9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:36.292885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.347878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.350110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.399057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1031/2063] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:36.402956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.467316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559abad4-7221-4f1b-b4d3-e5dcc57e48d7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.470142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:36.472745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.473605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.473820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.474099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] PATH_INFO: `/v3/projects/4e094a4a575b47b6a95cde5d386b0a9c` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.475372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e094a4a575b47b6a95cde5d386b0a9c)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.477509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.522515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bc828-1903-40a0-bd38-1eda2df6b1aa None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.530023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.531764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.532235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.532681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.533732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.551688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.559078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e094a4a575b47b6a95cde5d386b0a9c', 'request_id': 'req-40f0722a-87dc-452d-bb4d-ab20aab4b44c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:36.567670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e094a4a575b47b6a95cde5d386b0a9c', 'request_id': 'req-40f0722a-87dc-452d-bb4d-ab20aab4b44c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:36.580585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1032/2064] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:36.580775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e094a4a575b47b6a95cde5d386b0a9c', 'request_id': 'req-40f0722a-87dc-452d-bb4d-ab20aab4b44c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:36.584983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.602027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] The token cache is being invalidate because project 4e094a4a575b47b6a95cde5d386b0a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:36.602512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:36.603550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-40f0722a-87dc-452d-bb4d-ab20aab4b44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:36.605922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1033/2065] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:36 2022] DELETE /identity/v3/projects/4e094a4a575b47b6a95cde5d386b0a9c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:36.612040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb71fc-be3c-4585-829b-faef23baaae8 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.612421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb71fc-be3c-4585-829b-faef23baaae8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.618546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb71fc-be3c-4585-829b-faef23baaae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.638735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c7bb71fc-be3c-4585-829b-faef23baaae8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:36.640879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c7bb71fc-be3c-4585-829b-faef23baaae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.677328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:36.686412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9830f-535d-4e19-be35-06db913cf8c2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.693111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.693741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.694023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.694297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.695071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.697758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.726511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1033/2066] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:36.730456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.732279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.742316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1034/2067] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:36.748678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8987-5514-4dc7-b2c1-986d6beb9f45 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.750867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.751190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.751471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.755645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.756425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.756822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.757198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.758146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.760487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.773065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:36.773628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1034/2068] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:36.774678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.777888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.784405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1035/2069] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:36.788809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.793025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16eda29-c1d0-4c28-9c26-c74e261780e8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.798976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.799600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.799891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.800260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.800651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdb961-9046-4333-acf5-5b526a596307 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.800883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.802571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.807223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.808119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.808551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.808898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.809635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:36.810003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.811962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.818488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1035/2070] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:36.819751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.831312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1036/2071] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:36.832356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f221a-3780-4388-a120-19bbb716f931 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.835583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.837488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0933db-c9f8-42c5-a7d4-2770b4a66cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.838155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0933db-c9f8-42c5-a7d4-2770b4a66cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.838432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0933db-c9f8-42c5-a7d4-2770b4a66cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.838701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0933db-c9f8-42c5-a7d4-2770b4a66cbb admin admin] PATH_INFO: `/v3/projects/69f70078408b45368c665a918905be7c/users/751fad63ee094554940e130c7e2bfde4/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.847034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fb1bc-2d43-4b04-b915-edbfce426832 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.854663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.855391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.855699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.855977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.856069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0933db-c9f8-42c5-a7d4-2770b4a66cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f70078408b45368c665a918905be7c, user_id=751fad63ee094554940e130c7e2bfde4, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.856718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.859310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0933db-c9f8-42c5-a7d4-2770b4a66cbb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.859598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:36.873821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1036/2072] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:36 2022] PUT /identity/v3/projects/69f70078408b45368c665a918905be7c/users/751fad63ee094554940e130c7e2bfde4/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:36.890169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.891006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.891006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.902767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1037/2073] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:36.908141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:36.927341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] MFA Rules not processed for user `751fad63ee094554940e130c7e2bfde4`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:36.928767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:36.982491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:36.990251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa0086-e1cf-4c81-8188-6c7171851a05 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:36.995190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:36.995844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:36.996128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:36.996402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:36.997040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:36.999105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.004040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1037/2074] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:37.015745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.026192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1038/2075] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:36 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:37.027247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2977307-6645-44c1-99e1-733e5ba449f4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.030284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.032680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.033300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.033586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.033859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.034496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.036506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.040985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417328ca-8ebe-4428-87a4-97213276d596 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.052213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.052825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.053181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.053650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.054309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.055982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.068269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1039/2076] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:37.072231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1038/2077] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:37.074727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.074996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.085421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aea0f1-62bf-496c-8b79-2b77bf3b174d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.146046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232c975-7779-4af4-9fab-ba2378072a67 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.147631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.148355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.148703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.149049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.149764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.150914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.151522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.151632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.151821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.152095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.152729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.154658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.163921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1040/2078] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:37.167945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.175245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:37.178278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1039/2079] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:37.179429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd121-e98f-4411-bf42-e13fe2c83eb7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.182993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.194438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589439c-ed06-45c7-813b-57bf926dfe96 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.196819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10db463-5ebe-498e-ae7b-e32ed0fb1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.197657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db463-5ebe-498e-ae7b-e32ed0fb1152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.197942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db463-5ebe-498e-ae7b-e32ed0fb1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.198616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db463-5ebe-498e-ae7b-e32ed0fb1152 admin admin] PATH_INFO: `/v3/projects/e3755d85bb2e4a5daf34136ed20e3e0d/users/79baa5f3648549ab86bfd1ac61fb4e59/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.198802 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.199618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.199791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.199973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.200615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.202918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.214153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1040/2080] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:37.216061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10db463-5ebe-498e-ae7b-e32ed0fb1152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3755d85bb2e4a5daf34136ed20e3e0d, user_id=79baa5f3648549ab86bfd1ac61fb4e59, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.219069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.220176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10db463-5ebe-498e-ae7b-e32ed0fb1152 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.230882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01ee60-7bec-4609-9802-d60db8796dea admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.234815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1041/2081] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:37 2022] PUT /identity/v3/projects/e3755d85bb2e4a5daf34136ed20e3e0d/users/79baa5f3648549ab86bfd1ac61fb4e59/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:37.245353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.245642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.245919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.276489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] MFA Rules not processed for user `79baa5f3648549ab86bfd1ac61fb4e59`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:37.277764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:37.293619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.294291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.294570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.294867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] PATH_INFO: `/v3/projects/7a5e581b1303400496f05948f9490f62/users/70b54880a0074ba78c5591ca0484ce8b/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.311580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5e581b1303400496f05948f9490f62, user_id=70b54880a0074ba78c5591ca0484ce8b, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.314777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.333358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1041/2082] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:37 2022] PUT /identity/v3/projects/7a5e581b1303400496f05948f9490f62/users/70b54880a0074ba78c5591ca0484ce8b/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:37.337816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.354448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1042/2083] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:37.358593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.371970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688239-4db0-4fad-bcae-9c2f90e45c5b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.383512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:37.420504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b24376-b49d-43db-b9a5-37c4ee93eda2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.426722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.427452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.427809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.428210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.428958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.430857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.447291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.448852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.449165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.449449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.450328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.450821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1042/2084] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:37.453441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.458945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.469119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655af299-2424-4e41-aa3d-56746c8108d8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.475294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fecfd-8e77-43f5-9ec2-86990ff79fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.475967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fecfd-8e77-43f5-9ec2-86990ff79fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.476249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fecfd-8e77-43f5-9ec2-86990ff79fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.476529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fecfd-8e77-43f5-9ec2-86990ff79fe0 admin admin] PATH_INFO: `/v3/projects/7a5e581b1303400496f05948f9490f62/users/70b54880a0074ba78c5591ca0484ce8b/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.480608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fecfd-8e77-43f5-9ec2-86990ff79fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5e581b1303400496f05948f9490f62, user_id=70b54880a0074ba78c5591ca0484ce8b, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.484026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fecfd-8e77-43f5-9ec2-86990ff79fe0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.496684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1043/2085] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:37 2022] PUT /identity/v3/projects/7a5e581b1303400496f05948f9490f62/users/70b54880a0074ba78c5591ca0484ce8b/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:37.504489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1043/2086] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:37.508745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.509197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.509753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.509753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.540775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] MFA Rules not processed for user `70b54880a0074ba78c5591ca0484ce8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:37.542109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:37.573869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:37.579832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df214-ff71-4607-8679-404c2df77df2 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.586020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.586644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.587124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.587410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.588047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.589990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.612741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1044/2087] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:37.616591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.619104 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1044/2088] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:37.628619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a9411-fd3a-4670-a483-03c0f7f8a486 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.632306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.633383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.633775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:37.634152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.634439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.634714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.635407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.637041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.652784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1045/2089] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:37.654160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.666138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4b86-42ce-49ea-a1da-fae5d66624e6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.677301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.678004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.678591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.678591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] PATH_INFO: `/v3/projects/edd7d0632a8c41b3acd37c791974c454/users/bec2c50b0361497ca7c37b0657cd7447/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.697278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=edd7d0632a8c41b3acd37c791974c454, user_id=bec2c50b0361497ca7c37b0657cd7447, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.699496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec89c9-ff2b-4112-841a-ab01215286a5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.700417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.704487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.705112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.705395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.705672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.708271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:37.715411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1046/2090] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:37 2022] PUT /identity/v3/projects/edd7d0632a8c41b3acd37c791974c454/users/bec2c50b0361497ca7c37b0657cd7447/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:37.720673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.772966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:37.777456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.780150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.792368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1045/2091] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:00:37.796336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de984c-4ba6-45ee-ac4f-ebce585a5f0b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.801638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34658f84-46fa-4716-9367-c48acebf06a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.802288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34658f84-46fa-4716-9367-c48acebf06a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.802571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34658f84-46fa-4716-9367-c48acebf06a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.802856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34658f84-46fa-4716-9367-c48acebf06a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.803542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34658f84-46fa-4716-9367-c48acebf06a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.805173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34658f84-46fa-4716-9367-c48acebf06a5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.816943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1047/2092] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:37.821225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:37.840505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7bfd-cc22-434e-8385-89bce33e5a46 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:37.847080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:37.847774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.848059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.848484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] PATH_INFO: `/v3/projects/edd7d0632a8c41b3acd37c791974c454/users/bec2c50b0361497ca7c37b0657cd7447/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.850574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edd7d0632a8c41b3acd37c791974c454, user_id=bec2c50b0361497ca7c37b0657cd7447, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:37.854899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:37.870785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1046/2093] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:37 2022] PUT /identity/v3/projects/edd7d0632a8c41b3acd37c791974c454/users/bec2c50b0361497ca7c37b0657cd7447/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:37.884408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:37.884885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:37.885439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:37.916194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] MFA Rules not processed for user `bec2c50b0361497ca7c37b0657cd7447`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:37.917610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:38.017391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1048/2094] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:38.033952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:38.035930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:38.104399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a11b5-4714-4b7b-9e88-d4b2dd27d5e9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:38.108701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:38.109484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:38.110502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:38.110925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:38.123640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:38.124197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:38.136673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1047/2095] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:38.238774 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:38.248525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5e68c-7c7d-4682-a26d-59ab26635656 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:38.253243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:38.253920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:38.254240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:38.254549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:38.286943 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:38.317069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:38.318569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:38.328834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1049/2096] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:41.495461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:41.504729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955de2d6-45d9-496b-ae12-1c8b2fe6ab83 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:41.508313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:41.508895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:41.509175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:41.509448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:41.565359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:41.566749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:41.575847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1048/2097] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:42.218520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:42.235110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37583e12-8783-4ff8-81f0-8fc2bb34a345 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:42.239012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f267de-8ba1-4d34-b187-4d5895f9aa87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:42.239663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f267de-8ba1-4d34-b187-4d5895f9aa87 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:42.239919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f267de-8ba1-4d34-b187-4d5895f9aa87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:42.240192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f267de-8ba1-4d34-b187-4d5895f9aa87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:42.338554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f267de-8ba1-4d34-b187-4d5895f9aa87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:42.340032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f267de-8ba1-4d34-b187-4d5895f9aa87 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:42.352270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1050/2098] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:45.822009 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:45.936734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f39764-c712-480a-b6b8-034fb628e233 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:45.942730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:45.943483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:45.944708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:45.945203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] PATH_INFO: `/v3/users/79baa5f3648549ab86bfd1ac61fb4e59` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:45.946930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79baa5f3648549ab86bfd1ac61fb4e59)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:45.949575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:45.995916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:46.045629 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79baa5f3648549ab86bfd1ac61fb4e59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.047173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79baa5f3648549ab86bfd1ac61fb4e59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.047860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79baa5f3648549ab86bfd1ac61fb4e59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.055973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79baa5f3648549ab86bfd1ac61fb4e59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.069738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79baa5f3648549ab86bfd1ac61fb4e59'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.087433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1049/2099] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:45 2022] DELETE /identity/v3/users/79baa5f3648549ab86bfd1ac61fb4e59 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:46.091828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0a97-1b26-41ab-8550-f33302ff6472 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.092387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0a97-1b26-41ab-8550-f33302ff6472 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.092387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f0a97-1b26-41ab-8550-f33302ff6472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.114974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-315f0a97-1b26-41ab-8550-f33302ff6472 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:46.116783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-315f0a97-1b26-41ab-8550-f33302ff6472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.204189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1051/2100] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:46.216640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:46.218783 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.292159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb385f00-4f89-451e-ba50-4f9c0e5d8665 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:46.297463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9836d9-3f96-45d9-a463-e1f6119cdb86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:46.298082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9836d9-3f96-45d9-a463-e1f6119cdb86 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.298415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9836d9-3f96-45d9-a463-e1f6119cdb86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.298680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9836d9-3f96-45d9-a463-e1f6119cdb86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.309100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9836d9-3f96-45d9-a463-e1f6119cdb86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:46.310554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9836d9-3f96-45d9-a463-e1f6119cdb86 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:46.322393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1050/2101] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:46.539658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13f9c-511b-4474-a66f-3ef3b994b3c3 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.540283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13f9c-511b-4474-a66f-3ef3b994b3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.540768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee13f9c-511b-4474-a66f-3ef3b994b3c3 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.543803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1052/2102] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:00:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:00:46.552442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.552852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.553222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.577257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:46.579188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.645154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13f9c-511b-4474-a66f-3ef3b994b3c3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:46.657827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-8ee13f9c-511b-4474-a66f-3ef3b994b3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.681490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee13f9c-511b-4474-a66f-3ef3b994b3c3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:46.687564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:46.688545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.689156 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.689625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] PATH_INFO: `/v3/users/8d562e923af0446db67404ce4ad36cc7` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.691758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1051/2103] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:00:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:46.692437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:46.692793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d562e923af0446db67404ce4ad36cc7)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:46.694695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.696293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:46.713725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0c707-1e9e-418e-bdaa-787a68deee29 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:46.719124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:46.719824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.720187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.720522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] PATH_INFO: `/v3/projects/e3755d85bb2e4a5daf34136ed20e3e0d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.721717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3755d85bb2e4a5daf34136ed20e3e0d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:46.724651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:46.752705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d562e923af0446db67404ce4ad36cc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.753378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d562e923af0446db67404ce4ad36cc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.766285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d562e923af0446db67404ce4ad36cc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.768984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:46.779841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:46.785446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d562e923af0446db67404ce4ad36cc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.786243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d562e923af0446db67404ce4ad36cc7'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.797021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3755d85bb2e4a5daf34136ed20e3e0d', 'request_id': 'req-4395207c-67d2-405c-9f31-a9ba34ebd0dc'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.804907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3755d85bb2e4a5daf34136ed20e3e0d', 'request_id': 'req-4395207c-67d2-405c-9f31-a9ba34ebd0dc'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.809537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1053/2104] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:46 2022] DELETE /identity/v3/users/8d562e923af0446db67404ce4ad36cc7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:46.809537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:46.810059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.821118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3755d85bb2e4a5daf34136ed20e3e0d', 'request_id': 'req-4395207c-67d2-405c-9f31-a9ba34ebd0dc'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.848317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] The token cache is being invalidate because project e3755d85bb2e4a5daf34136ed20e3e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:46.849653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.850587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:46.852385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1052/2105] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:46 2022] DELETE /identity/v3/projects/e3755d85bb2e4a5daf34136ed20e3e0d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:46.856827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:46.860458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.924611 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986933ea-faad-4fa6-b42a-385fb9fb05ab admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:46.931476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:46.931830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:46.932185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:46.932529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:46.934426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:46.950315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:46.952162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:46.958271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395207c-67d2-405c-9f31-a9ba34ebd0dc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:46.963667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1054/2106] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:47.027086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.027808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.028099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.028376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] PATH_INFO: `/v3/users/bec2c50b0361497ca7c37b0657cd7447` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.029711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec2c50b0361497ca7c37b0657cd7447)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.032887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.040772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.042344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:47.065997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc698-2554-471e-a482-93e880f32cb5 service placement] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.072231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.072907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.073454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.073769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] PATH_INFO: `/v3/projects/aa42868f9e9e4d00be6a78ec54924712` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.075092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa42868f9e9e4d00be6a78ec54924712)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.077882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.087942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec2c50b0361497ca7c37b0657cd7447'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.088649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec2c50b0361497ca7c37b0657cd7447'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.089280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec2c50b0361497ca7c37b0657cd7447'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.102766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec2c50b0361497ca7c37b0657cd7447'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.115267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:47.117302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c4e9281f-1808-4967-b1b3-3d4f36f43a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec2c50b0361497ca7c37b0657cd7447'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.136400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1053/2107] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:46 2022] DELETE /identity/v3/users/bec2c50b0361497ca7c37b0657cd7447 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:47.158613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa42868f9e9e4d00be6a78ec54924712', 'request_id': 'req-2baa9e0b-3a06-4e00-b26d-735a9c27783c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.167321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa42868f9e9e4d00be6a78ec54924712', 'request_id': 'req-2baa9e0b-3a06-4e00-b26d-735a9c27783c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.176018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa42868f9e9e4d00be6a78ec54924712', 'request_id': 'req-2baa9e0b-3a06-4e00-b26d-735a9c27783c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.194657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] The token cache is being invalidate because project aa42868f9e9e4d00be6a78ec54924712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:47.195014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.195764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2baa9e0b-3a06-4e00-b26d-735a9c27783c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.197708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1055/2108] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:47 2022] DELETE /identity/v3/projects/aa42868f9e9e4d00be6a78ec54924712 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:47.209588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.210031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.210405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.231663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:47.232924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:47.302256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1054/2109] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:47.310368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.310767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.311091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.332597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:47.333822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:47.341915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1056/2110] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:47.346619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.356355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b99ea3-13d4-44b1-bb59-b2d9ef3837e4 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.362035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.362653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.362962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.363235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.363904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:47.364187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.365892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.382523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1055/2111] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:47.383956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.394173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4eb4a-4a3b-4991-849c-696fc00a3402 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.398977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.399617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.399898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.400171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.400803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.402707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.426087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.427868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:47.435275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1057/2112] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:47.436541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.442752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e4979-d523-4aba-9522-662544a9010a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.468018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:47.512402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.513157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.513439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.513851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] PATH_INFO: `/v3/projects/edd7d0632a8c41b3acd37c791974c454` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.514290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d150fd-0a1b-4419-acfd-3b96df6c6341 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.515741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edd7d0632a8c41b3acd37c791974c454)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.517648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.519960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.520573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.520853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.521128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.521758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.523703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.545708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1058/2113] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:47.550338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.559895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464914d0-cac5-4ef1-9cac-73d35f37c498 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.573721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edd7d0632a8c41b3acd37c791974c454', 'request_id': 'req-c6905776-9cce-4e6f-bd00-62edf68b0775'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.581628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edd7d0632a8c41b3acd37c791974c454', 'request_id': 'req-c6905776-9cce-4e6f-bd00-62edf68b0775'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.589132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edd7d0632a8c41b3acd37c791974c454', 'request_id': 'req-c6905776-9cce-4e6f-bd00-62edf68b0775'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.602219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] The token cache is being invalidate because project edd7d0632a8c41b3acd37c791974c454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:47.602511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.603188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:47.605161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1056/2114] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:47 2022] DELETE /identity/v3/projects/edd7d0632a8c41b3acd37c791974c454 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:47.630351 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.631015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.631300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.631577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.632237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.633885 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.645476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1059/2115] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:47.649958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.651676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:47.664528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6905776-9cce-4e6f-bd00-62edf68b0775 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.669468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8359f-5155-4924-ba42-26f877243355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.670075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8359f-5155-4924-ba42-26f877243355 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.670566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8359f-5155-4924-ba42-26f877243355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.670674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8359f-5155-4924-ba42-26f877243355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.672007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8359f-5155-4924-ba42-26f877243355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.673987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8359f-5155-4924-ba42-26f877243355 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.675779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:47.691839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1057/2116] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:47.692977 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.705455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3b29b-3e28-4823-94dc-68d0d2aa2cf4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.710596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.711344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.711631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.711907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] PATH_INFO: `/v3/projects/ee0472eaea3e489498023f3d3ccbdb19/users/f419d441cfb8486896fb0fa93218078e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.729134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee0472eaea3e489498023f3d3ccbdb19, user_id=f419d441cfb8486896fb0fa93218078e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.731904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.747593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1060/2117] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:47 2022] PUT /identity/v3/projects/ee0472eaea3e489498023f3d3ccbdb19/users/f419d441cfb8486896fb0fa93218078e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:47.758676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.758995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.759272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.789962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] MFA Rules not processed for user `f419d441cfb8486896fb0fa93218078e`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:47.791317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:47.862680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1058/2118] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:47.867266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.932879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e630b0-0082-4183-ab94-e09552091a20 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:47.938501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:47.939149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:47.939454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:47.939726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:47.940369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:47.942320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:47.971008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1061/2119] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:47.974700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:47.986412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357337c-b16b-4d41-ad9e-fd9ca5e6ae91 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.041920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.042519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.042794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.043092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.043722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.045628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.067759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1059/2120] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:47 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:48.071612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:48.082330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92037213-5224-4a36-81ce-66eff1fc8908 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.086814 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.087445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.087723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.087996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.088636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.090217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.091462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:48.102938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1062/2121] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:48.106942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:48.117651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b21b61-5afd-4a0f-add8-913fc2b23ac2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.122317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.123059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.123405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.123697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] PATH_INFO: `/v3/projects/5152af4759134d44ab4a8e077c8393a5/users/42a3cb7887584b74a933373f1cc209c4/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.138196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5152af4759134d44ab4a8e077c8393a5, user_id=42a3cb7887584b74a933373f1cc209c4, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.141001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.153742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1060/2122] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:48 2022] PUT /identity/v3/projects/5152af4759134d44ab4a8e077c8393a5/users/42a3cb7887584b74a933373f1cc209c4/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:48.159149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:48.171712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70bbfa-d5dd-4128-b75e-b32d692a43e9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.238796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47388f22-843d-489c-bff8-9a9926414678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.239467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47388f22-843d-489c-bff8-9a9926414678 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.239773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47388f22-843d-489c-bff8-9a9926414678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.240072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47388f22-843d-489c-bff8-9a9926414678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.240764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47388f22-843d-489c-bff8-9a9926414678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.242500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47388f22-843d-489c-bff8-9a9926414678 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.253222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1063/2123] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:48.257412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:48.267650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95fd82-87ac-4703-8af8-91bd33bdc4ad admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.272198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.272871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.273160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.273431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] PATH_INFO: `/v3/projects/5152af4759134d44ab4a8e077c8393a5/users/42a3cb7887584b74a933373f1cc209c4/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.275349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5152af4759134d44ab4a8e077c8393a5, user_id=42a3cb7887584b74a933373f1cc209c4, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.277966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.288497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1061/2124] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:48 2022] PUT /identity/v3/projects/5152af4759134d44ab4a8e077c8393a5/users/42a3cb7887584b74a933373f1cc209c4/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:48.300703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.300994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.301272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.329576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] MFA Rules not processed for user `42a3cb7887584b74a933373f1cc209c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:48.330809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:48.399486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1064/2125] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:48.411767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:48.413520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:48.456745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:48.473533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bed8f-183c-4f20-91d5-a5d62d8ed1c4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.477367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.477980 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.478256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.478530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.488655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.489880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.500987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1062/2126] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:48.596516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:48.613102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be37d-cb43-4dd3-8398-f003acfcea29 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:48.618700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ab0ce-310b-49ac-8b28-0cb26e6dc04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:48.619746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ab0ce-310b-49ac-8b28-0cb26e6dc04a service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:48.620205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ab0ce-310b-49ac-8b28-0cb26e6dc04a service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:48.620669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ab0ce-310b-49ac-8b28-0cb26e6dc04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:48.716904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ab0ce-310b-49ac-8b28-0cb26e6dc04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:48.718952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ab0ce-310b-49ac-8b28-0cb26e6dc04a service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:48.728253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1065/2127] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:49.785543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:49.787014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:49.856316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62cdbd-1d0c-4707-b06e-d1cb8b9dc2f0 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:49.862081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:49.862722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:49.863060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:49.863314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] PATH_INFO: `/v3/users/11be0eea7b634533a9b72dfd5d76b9a7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:49.864466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] RBAC: Authorizing `identity:delete_user(user_id=11be0eea7b634533a9b72dfd5d76b9a7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:49.866399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:49.925352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11be0eea7b634533a9b72dfd5d76b9a7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:49.926198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11be0eea7b634533a9b72dfd5d76b9a7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:49.926806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11be0eea7b634533a9b72dfd5d76b9a7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:49.934243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11be0eea7b634533a9b72dfd5d76b9a7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:49.950667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11be0eea7b634533a9b72dfd5d76b9a7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:49.974744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1063/2128] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:49 2022] DELETE /identity/v3/users/11be0eea7b634533a9b72dfd5d76b9a7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:49.979191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:49.979633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:49.979763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:50.017331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:50.018753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:50.035764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:50.114817 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1066/2129] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:50.127633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:50.132526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:50.301484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649983d-4067-4aab-9685-68dc862ceaee admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:50.305847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:50.306465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:50.306744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:50.307050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:50.320148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:50.322414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:50.343075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1064/2130] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:50.826081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:50.828389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:50.842703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd9173-7c92-4cf5-8c48-4f779e2b9c09 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:50.848549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:50.849142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:50.849413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:50.849687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] PATH_INFO: `/v3/projects/b69abb36de2141628e74868c582e079d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:50.850858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69abb36de2141628e74868c582e079d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:50.852844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:50.916794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69abb36de2141628e74868c582e079d', 'request_id': 'req-964e4d87-c08f-439e-84b5-f6fd690406dd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:50.924190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69abb36de2141628e74868c582e079d', 'request_id': 'req-964e4d87-c08f-439e-84b5-f6fd690406dd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:50.931422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69abb36de2141628e74868c582e079d', 'request_id': 'req-964e4d87-c08f-439e-84b5-f6fd690406dd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:50.953701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] The token cache is being invalidate because project b69abb36de2141628e74868c582e079d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:50.954203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:50.955416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:50.957033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1067/2131] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:50 2022] DELETE /identity/v3/projects/b69abb36de2141628e74868c582e079d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:50.961155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:50.972580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:50.987790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ada11-bf6a-407c-bbce-1bd228c54434 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:51.047799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:51.048473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:51.048759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:51.049039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] PATH_INFO: `/v3/users/751fad63ee094554940e130c7e2bfde4` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:51.050193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=751fad63ee094554940e130c7e2bfde4)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:51.052201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:51.105508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751fad63ee094554940e130c7e2bfde4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.106198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751fad63ee094554940e130c7e2bfde4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.106786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751fad63ee094554940e130c7e2bfde4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.116334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751fad63ee094554940e130c7e2bfde4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.129819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751fad63ee094554940e130c7e2bfde4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.146000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1065/2132] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:50 2022] DELETE /identity/v3/users/751fad63ee094554940e130c7e2bfde4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:51.466454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:51.468649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:51.491863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e4d87-c08f-439e-84b5-f6fd690406dd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:51.600831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:51.602041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:51.602668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:51.603226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] PATH_INFO: `/v3/projects/69f70078408b45368c665a918905be7c` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:51.604526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f70078408b45368c665a918905be7c)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:51.606561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:51.701385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:51.713040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f70078408b45368c665a918905be7c', 'request_id': 'req-d6172134-a98c-4335-a73e-b3817e70385e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.724627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f70078408b45368c665a918905be7c', 'request_id': 'req-d6172134-a98c-4335-a73e-b3817e70385e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.739041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f70078408b45368c665a918905be7c', 'request_id': 'req-d6172134-a98c-4335-a73e-b3817e70385e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.760714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] The token cache is being invalidate because project 69f70078408b45368c665a918905be7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:51.760714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.761659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.763282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1068/2133] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:51 2022] DELETE /identity/v3/projects/69f70078408b45368c665a918905be7c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:51.770031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:51.772201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:51.787934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460bd05-4648-4966-9129-f385ca700e9f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:51.863541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:51.864239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:51.865116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:51.865393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] PATH_INFO: `/v3/users/70b54880a0074ba78c5591ca0484ce8b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:51.866569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b54880a0074ba78c5591ca0484ce8b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:51.869187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:51.901062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:51.928634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b54880a0074ba78c5591ca0484ce8b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.929830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b54880a0074ba78c5591ca0484ce8b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.930440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b54880a0074ba78c5591ca0484ce8b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.937709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b54880a0074ba78c5591ca0484ce8b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.951457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-25a4e87c-5338-4dcf-b60f-64eb1f553a32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b54880a0074ba78c5591ca0484ce8b'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:51.966683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1066/2134] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:51 2022] DELETE /identity/v3/users/70b54880a0074ba78c5591ca0484ce8b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:52.269390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.270895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:52.284974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172134-a98c-4335-a73e-b3817e70385e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.347339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.347959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.348374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.348699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] PATH_INFO: `/v3/projects/7a5e581b1303400496f05948f9490f62` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.349974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5e581b1303400496f05948f9490f62)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.352045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.404417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5e581b1303400496f05948f9490f62', 'request_id': 'req-71a0dbbc-422f-4651-8e3f-5474444be49b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:52.411204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5e581b1303400496f05948f9490f62', 'request_id': 'req-71a0dbbc-422f-4651-8e3f-5474444be49b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:52.417613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5e581b1303400496f05948f9490f62', 'request_id': 'req-71a0dbbc-422f-4651-8e3f-5474444be49b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:52.432052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] The token cache is being invalidate because project 7a5e581b1303400496f05948f9490f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:52.432342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:52.432997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-71a0dbbc-422f-4651-8e3f-5474444be49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:52.434526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1069/2135] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:52 2022] DELETE /identity/v3/projects/7a5e581b1303400496f05948f9490f62 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:52.440491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.440778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.441051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.461811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:52.463064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:52.529228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1067/2136] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:52.537757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.538317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.538411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.563687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:52.565048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:52.574126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1070/2137] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:00:52.579359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.589643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d4e76-ced7-4e76-8cef-19cc1f6a4f24 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.595095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.595563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.595873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.596145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.596796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:52.597099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.598775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.611239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1068/2138] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:00:52.615843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.625809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c784140-563f-40aa-8624-14dbe13298a7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.630502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.631143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.631419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.631689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.632330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.634212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.664083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1071/2139] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:52 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:52.668286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.679807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6200b-4e23-47b0-b85b-0b52ea220a6e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.738268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.738899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.739181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.739450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.740092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.742014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.768299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1069/2140] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:52 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:00:52.772353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.782859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d30173-7feb-41d0-bf83-7a00723e910b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.787864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.788480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.788757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.789025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.789693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.791368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.805914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1072/2141] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:52.807262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.818608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb805-1115-43d3-baa6-807c47619d5a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.823683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fe607-3800-40c7-a34f-72348385209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.824295 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe607-3800-40c7-a34f-72348385209f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.824577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe607-3800-40c7-a34f-72348385209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.824941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe607-3800-40c7-a34f-72348385209f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.825571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fe607-3800-40c7-a34f-72348385209f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.827861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fe607-3800-40c7-a34f-72348385209f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.841378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1070/2142] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:00:52.843718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:52.854041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed020-39ec-4450-b7f3-5b4fbfdbeca2 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:52.867443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:52.868129 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.868410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.868683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] PATH_INFO: `/v3/projects/c804c82194c740b3906c94b9c0ceccbe/users/ed5c45f5a7e4430e8e20ffe93f6d7807/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.889501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c804c82194c740b3906c94b9c0ceccbe, user_id=ed5c45f5a7e4430e8e20ffe93f6d7807, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:52.894528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:52.902382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:52.913539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1073/2143] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:52 2022] PUT /identity/v3/projects/c804c82194c740b3906c94b9c0ceccbe/users/ed5c45f5a7e4430e8e20ffe93f6d7807/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:52.925854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:52.926139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:52.926424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:52.942139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:52.960727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] MFA Rules not processed for user `ed5c45f5a7e4430e8e20ffe93f6d7807`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:52.961969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:53.047514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1071/2144] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:53.060429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:53.064248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:53.141127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e2572-264a-4e30-9045-350d5c8f540b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:53.144813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4de21-d48b-43fc-8de5-f12401ecdbb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:53.145404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4de21-d48b-43fc-8de5-f12401ecdbb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:53.145681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4de21-d48b-43fc-8de5-f12401ecdbb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:53.145947 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4de21-d48b-43fc-8de5-f12401ecdbb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:53.154590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4de21-d48b-43fc-8de5-f12401ecdbb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:53.156662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4de21-d48b-43fc-8de5-f12401ecdbb1 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:53.165754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1074/2145] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:58.036290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:58.037919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:58.113874 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c802f-1ff8-43fb-88cd-1afeea93589c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:58.118821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:58.119524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:58.119803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:58.120080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] PATH_INFO: `/v3/users/c655248e25a14d9dbe4d5c491b688991` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:58.121431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c655248e25a14d9dbe4d5c491b688991)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:58.123316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:58.192388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c655248e25a14d9dbe4d5c491b688991'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:58.193092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c655248e25a14d9dbe4d5c491b688991'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:58.193717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c655248e25a14d9dbe4d5c491b688991'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:58.204706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c655248e25a14d9dbe4d5c491b688991'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:58.221617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c655248e25a14d9dbe4d5c491b688991'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:58.241846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1072/2146] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:00:58 2022] DELETE /identity/v3/users/c655248e25a14d9dbe4d5c491b688991 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:58.248868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:58.250995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:58.250995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:58.284672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:58.286502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:58.392262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1075/2147] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:58.408664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:58.410909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:58.438372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:58.500815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073b255-3a90-493f-ac59-ee6a2a0868be admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:58.504603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0f512-3037-4f9f-bb1b-05cefb21ec72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:58.505265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0f512-3037-4f9f-bb1b-05cefb21ec72 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:58.505541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0f512-3037-4f9f-bb1b-05cefb21ec72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:58.505812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0f512-3037-4f9f-bb1b-05cefb21ec72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:58.520016 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0f512-3037-4f9f-bb1b-05cefb21ec72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:58.521528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0f512-3037-4f9f-bb1b-05cefb21ec72 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:58.535431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1073/2148] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:00:58.943309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:58.946680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:58.972907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea04f7-85c6-43c3-bcf5-b60babef55a4 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:58.985982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:58.987548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:58.988248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:58.988672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] PATH_INFO: `/v3/projects/33baca214d8f4909aa19736491569b30` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:58.991748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33baca214d8f4909aa19736491569b30)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:58.994295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.076555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33baca214d8f4909aa19736491569b30', 'request_id': 'req-a759ebcd-5b98-487c-863b-1010eb4a6ee0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:59.087136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33baca214d8f4909aa19736491569b30', 'request_id': 'req-a759ebcd-5b98-487c-863b-1010eb4a6ee0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:59.097407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33baca214d8f4909aa19736491569b30', 'request_id': 'req-a759ebcd-5b98-487c-863b-1010eb4a6ee0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:59.126223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] The token cache is being invalidate because project 33baca214d8f4909aa19736491569b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:00:59.127504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:59.128050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a759ebcd-5b98-487c-863b-1010eb4a6ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:00:59.138640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1076/2149] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:00:58 2022] DELETE /identity/v3/projects/33baca214d8f4909aa19736491569b30 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:00:59.141545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.142314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.142493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.176682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:59.178907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:59.379474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1074/2150] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:59.393174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.393968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.394115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.425168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:00:59.427553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:00:59.441984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1077/2151] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:00:59.447870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:59.461762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7f17d-3d86-4a4c-b081-8f827f82d12c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:59.470482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:59.471328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.471683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.471916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.472627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:00:59.472975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:59.475121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.497789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1075/2152] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:00:59.499968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:59.512230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea2ffc-6217-47ba-9e80-e77c9a5d31b3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:59.520471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:59.521452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.521909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.522357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.523420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:59.525526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.549913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:00:59.573879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1078/2153] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:00:59 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:59.578739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:59.590148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4467901-ebe8-4f10-9802-42fc3ee8c14a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:59.707080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:59.707907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.708227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.708524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.709102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:59.713659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.750532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1076/2154] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:00:59 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:00:59.756589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:59.768636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70f3e5-18d0-46f1-acea-582516f16d07 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:59.777301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:59.778020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.778369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.778713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.779523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:59.782815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.797625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1079/2155] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:59.804356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:59.817352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63987a-8133-48a3-b779-b05c20738ed1 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:59.823862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d283ec-bee6-4803-a3cf-3d7a4cf4fc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:59.824614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d283ec-bee6-4803-a3cf-3d7a4cf4fc94 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.825029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d283ec-bee6-4803-a3cf-3d7a4cf4fc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.825376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d283ec-bee6-4803-a3cf-3d7a4cf4fc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.826187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d283ec-bee6-4803-a3cf-3d7a4cf4fc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:59.828667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d283ec-bee6-4803-a3cf-3d7a4cf4fc94 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.842172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1077/2156] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:00:59.847480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:00:59.860120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35eb53-08c9-45c6-b752-25ba1603283a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:00:59.870542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:00:59.872230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.872702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.873194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] PATH_INFO: `/v3/projects/9d0160f2201546a385be13aea0454fdc/users/9979be70f604408d9b78050e3ba52f6e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:00:59.902342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0160f2201546a385be13aea0454fdc, user_id=9979be70f604408d9b78050e3ba52f6e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:00:59.908004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:00:59.941046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1080/2157] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:00:59 2022] PUT /identity/v3/projects/9d0160f2201546a385be13aea0454fdc/users/9979be70f604408d9b78050e3ba52f6e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:00:59.946301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:00:59.946614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:00:59.946987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:00.003203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:00.005120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:00.033758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:00.113529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1078/2158] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:01:00.126427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:00.128340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:00.206514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecee9cd-4fe2-4029-8c6a-f78e7dabee45 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:00.210584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b87c9-9576-4da8-a44d-f79d0e49d53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:00.211238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b87c9-9576-4da8-a44d-f79d0e49d53b service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:00.211518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b87c9-9576-4da8-a44d-f79d0e49d53b service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:00.211801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b87c9-9576-4da8-a44d-f79d0e49d53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:00.226905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b87c9-9576-4da8-a44d-f79d0e49d53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:00.229644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b87c9-9576-4da8-a44d-f79d0e49d53b service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:00.245462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1081/2159] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:06.431735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:06.437703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:06.488929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbbad4-4ce1-417b-9427-11e5cde2055a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:06.508141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:06.508797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:06.509082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:06.509358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] PATH_INFO: `/v3/users/ed5c45f5a7e4430e8e20ffe93f6d7807` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:06.510654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5c45f5a7e4430e8e20ffe93f6d7807)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:06.512776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:06.640211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5c45f5a7e4430e8e20ffe93f6d7807'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:06.641660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5c45f5a7e4430e8e20ffe93f6d7807'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:06.643107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5c45f5a7e4430e8e20ffe93f6d7807'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:06.653109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5c45f5a7e4430e8e20ffe93f6d7807'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:06.701602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e32af14f-efca-4934-9af3-66d9f95cf17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5c45f5a7e4430e8e20ffe93f6d7807'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:06.735614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1079/2160] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:06 2022] DELETE /identity/v3/users/ed5c45f5a7e4430e8e20ffe93f6d7807 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:06.741678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:06.742437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:06.742437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:06.781672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:06.784561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:06.910553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:06.910986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:06.911142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:06.940497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1082/2161] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:06.964705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:06.966743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:06.978518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] MFA Rules not processed for user `9979be70f604408d9b78050e3ba52f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:06.982616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:07.161749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a67d2-58e4-45a9-be2c-4af6eb42da99 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.168012 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.168729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.169174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.169600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.183159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1080/2162] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:07.184516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.186011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.188742 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.190238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:07.210507 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1083/2163] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:01:07.213260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e5840-4efd-4582-9ffd-7c1f194688ad None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.221648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.222301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.222593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.222909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.224213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.226954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.302705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1081/2164] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:07 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:01:07.307080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.320081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:07.329066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04880ea2-feb4-47ce-95e6-9364e5d0c314 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.429544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.430070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.433403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.433881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.434511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.437408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.466619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1084/2165] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:07 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:01:07.473302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.485087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:07.491902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6294e1-cebb-4912-bf16-aa6dc424ce89 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.501708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.502961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.503517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.504019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.505230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.508356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.523510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1082/2166] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:01:07.528736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.541061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1123f5-910c-4fbb-ba89-6aa487dd0486 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.548085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.548783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.549061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.549335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] PATH_INFO: `/v3/projects/c1c777eae8de44b18cae19cd7d9d8cbc/users/f928305bc4ac4ba4a35b0893d963ac8f/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.571071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c777eae8de44b18cae19cd7d9d8cbc, user_id=f928305bc4ac4ba4a35b0893d963ac8f, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.576872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.597178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1085/2167] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:07 2022] PUT /identity/v3/projects/c1c777eae8de44b18cae19cd7d9d8cbc/users/f928305bc4ac4ba4a35b0893d963ac8f/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:01:07.601644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.625955 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04b050-797b-4b30-8100-f28be7d3a90f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.657516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.665953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:07.750753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.752953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.753439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.763894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.766303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.769942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.815038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc273afb-d9bc-4ff8-bd18-09dbcce30577 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.843756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.844560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.844822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1083/2168] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:07.846212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.846511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] PATH_INFO: `/v3/projects/c804c82194c740b3906c94b9c0ceccbe` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.849592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c804c82194c740b3906c94b9c0ceccbe)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.852684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.857373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.888666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314472b-ed50-49bc-9a3d-970a24f7b944 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.898259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2edd2-515a-4dc4-ab45-712fe387f963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.899345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2edd2-515a-4dc4-ab45-712fe387f963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.899786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2edd2-515a-4dc4-ab45-712fe387f963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.900255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2edd2-515a-4dc4-ab45-712fe387f963 admin admin] PATH_INFO: `/v3/projects/c1c777eae8de44b18cae19cd7d9d8cbc/users/f928305bc4ac4ba4a35b0893d963ac8f/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.903684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2edd2-515a-4dc4-ab45-712fe387f963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c777eae8de44b18cae19cd7d9d8cbc, user_id=f928305bc4ac4ba4a35b0893d963ac8f, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.914219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2edd2-515a-4dc4-ab45-712fe387f963 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.947514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1084/2169] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:07 2022] PUT /identity/v3/projects/c1c777eae8de44b18cae19cd7d9d8cbc/users/f928305bc4ac4ba4a35b0893d963ac8f/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:08.046673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c804c82194c740b3906c94b9c0ceccbe', 'request_id': 'req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:08.062137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c804c82194c740b3906c94b9c0ceccbe', 'request_id': 'req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:08.089036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c804c82194c740b3906c94b9c0ceccbe', 'request_id': 'req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:08.112693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:08.118670 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] The token cache is being invalidate because project c804c82194c740b3906c94b9c0ceccbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:08.119277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:08.123655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:08.126750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1086/2170] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:07 2022] DELETE /identity/v3/projects/c804c82194c740b3906c94b9c0ceccbe => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:15.305270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00acf5-a998-4976-8d95-512d6d264c68 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:15.306011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00acf5-a998-4976-8d95-512d6d264c68 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:15.306011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00acf5-a998-4976-8d95-512d6d264c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:15.353383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-1a00acf5-a998-4976-8d95-512d6d264c68 None None] MFA Rules not processed for user `f928305bc4ac4ba4a35b0893d963ac8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:15.355064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1a00acf5-a998-4976-8d95-512d6d264c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:15.475122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1085/2171] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:01:15.486583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:15.488089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:15.617140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdaa8d-f05f-42b5-bb48-12c9fe0f67c8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:15.625605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:15.626585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:15.627227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:15.627227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:15.630321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:15.725931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:15.727772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5384c67-866f-4f6e-8963-78a27b5512ff service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:15.739864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1087/2172] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:25.205437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:25.205934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:25.206632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:25.231429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] MFA Rules not processed for user `008e6c2c897741889510e6bf5181a9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:25.232659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:25.297020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1086/2173] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:25.733431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=56 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:35.287764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:35.958256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59264ada-4afd-4a88-84c2-adf97117b187 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:35.958790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59264ada-4afd-4a88-84c2-adf97117b187 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:35.958924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59264ada-4afd-4a88-84c2-adf97117b187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:35.991881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-59264ada-4afd-4a88-84c2-adf97117b187 None None] MFA Rules not processed for user `276174ef02b1409b840a0de13aa4095d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:35.993044 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-59264ada-4afd-4a88-84c2-adf97117b187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:36.052319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1088/2174] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:37.140632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:37.142867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:37.234965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44d802-ef66-4fb6-8ccc-30a285d3c3e9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:37.241551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:37.242499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:37.243005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:37.243273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] PATH_INFO: `/v3/users/f419d441cfb8486896fb0fa93218078e` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:37.244621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f419d441cfb8486896fb0fa93218078e)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:37.246583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:37.342424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f419d441cfb8486896fb0fa93218078e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:37.343817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f419d441cfb8486896fb0fa93218078e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:37.344490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f419d441cfb8486896fb0fa93218078e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:37.359065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f419d441cfb8486896fb0fa93218078e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:37.383106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f419d441cfb8486896fb0fa93218078e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:37.401886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1087/2175] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:37 2022] DELETE /identity/v3/users/f419d441cfb8486896fb0fa93218078e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:37.406272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:37.406822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:37.407054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:37.430863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:37.432103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:37.497428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1089/2176] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:37.508756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:37.510227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:37.575501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef101f-e8f6-4336-9040-a0608016188f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:37.580074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:37.581615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:37.581948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:37.582331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:37.594152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:37.595849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:37.606306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1088/2177] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:37.998384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:38.004107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:38.043994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3ba87-b498-477b-b435-c06e9e38c4a7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:38.052503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:38.053016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:38.053303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:38.053581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] PATH_INFO: `/v3/projects/ee0472eaea3e489498023f3d3ccbdb19` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:38.054759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee0472eaea3e489498023f3d3ccbdb19)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:38.056791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:38.122407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee0472eaea3e489498023f3d3ccbdb19', 'request_id': 'req-631c9e9f-0c98-4d95-93c8-7d708da523b9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.129359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee0472eaea3e489498023f3d3ccbdb19', 'request_id': 'req-631c9e9f-0c98-4d95-93c8-7d708da523b9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.136332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee0472eaea3e489498023f3d3ccbdb19', 'request_id': 'req-631c9e9f-0c98-4d95-93c8-7d708da523b9'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.150765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] The token cache is being invalidate because project ee0472eaea3e489498023f3d3ccbdb19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:38.151097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.151784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.153420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1090/2178] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:37 2022] DELETE /identity/v3/projects/ee0472eaea3e489498023f3d3ccbdb19 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:38.158767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:38.170575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55a2d-c3c5-4560-8714-cfd18f3fbcce service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:38.195677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:38.249255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:38.249998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:38.250343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:38.250678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] PATH_INFO: `/v3/users/42a3cb7887584b74a933373f1cc209c4` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:38.252581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a3cb7887584b74a933373f1cc209c4)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:38.254610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:38.306702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a3cb7887584b74a933373f1cc209c4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.307407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a3cb7887584b74a933373f1cc209c4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.308018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a3cb7887584b74a933373f1cc209c4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.317694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a3cb7887584b74a933373f1cc209c4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.340339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c112cb71-626b-448b-8f67-9150938d8e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a3cb7887584b74a933373f1cc209c4'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.379156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1089/2179] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:38 2022] DELETE /identity/v3/users/42a3cb7887584b74a933373f1cc209c4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:38.665191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:38.666732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:38.686337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c9e9f-0c98-4d95-93c8-7d708da523b9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:38.719476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:38.756993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:38.757636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:38.757915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:38.758185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] PATH_INFO: `/v3/projects/5152af4759134d44ab4a8e077c8393a5` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:38.759904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5152af4759134d44ab4a8e077c8393a5)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:38.761830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:38.816408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5152af4759134d44ab4a8e077c8393a5', 'request_id': 'req-ecc9515c-34b6-4638-bbfd-770e348e30e1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.822321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5152af4759134d44ab4a8e077c8393a5', 'request_id': 'req-ecc9515c-34b6-4638-bbfd-770e348e30e1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.828990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5152af4759134d44ab4a8e077c8393a5', 'request_id': 'req-ecc9515c-34b6-4638-bbfd-770e348e30e1'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.842746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] The token cache is being invalidate because project 5152af4759134d44ab4a8e077c8393a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:38.844059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.844059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ecc9515c-34b6-4638-bbfd-770e348e30e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:38.845412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1091/2180] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:38 2022] DELETE /identity/v3/projects/5152af4759134d44ab4a8e077c8393a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:38.850446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:38.850732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:38.851050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:38.871915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:38.873342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:38.951556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1090/2181] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:38.960219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:38.961350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:38.961743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:38.994730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:38.996034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:39.004902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1092/2182] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:39.009065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.019540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56915a4-202e-4da1-8255-1b507e4db81c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.024459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.025165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.025445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.025720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.026376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:01:39.026680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.028703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.052740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1091/2183] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:01:39.056787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.069753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b0db-1844-436c-84cc-43948b0aab6d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.081172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.081172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.081495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.081621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.082300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.084262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.140673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1093/2184] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:39 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:01:39.146855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.158593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747f5f1-ef6a-4745-b2be-f4a022f1cfbc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.195574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:39.243984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.244606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.244883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.245155 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.245807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.249008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.273157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1092/2185] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:39 2022] POST /identity/v3/users => generated 283 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:39.277128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.291152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f7bb7-1409-4717-9b9d-0786daacc2d9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.296005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.296626 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.296912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.297364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.298131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.299714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.314972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1094/2186] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:39.316243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.327697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc9d0-9f61-47cb-99b9-ab44185a8a64 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.334914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13571549-8dd2-4bd2-a969-67d526375239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.335900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13571549-8dd2-4bd2-a969-67d526375239 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.336395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13571549-8dd2-4bd2-a969-67d526375239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.336877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13571549-8dd2-4bd2-a969-67d526375239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.337738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13571549-8dd2-4bd2-a969-67d526375239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.339673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13571549-8dd2-4bd2-a969-67d526375239 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.358288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1093/2187] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:39.359650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.372863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603656da-3da7-47f6-b66f-5e7fe11c4c20 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.378625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.379405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.379689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.381072 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] PATH_INFO: `/v3/projects/590007d5ba314dfb8df70c171578a238/users/4b5d5ac20e96469cadb7fc92f62f1c01/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.415326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=590007d5ba314dfb8df70c171578a238, user_id=4b5d5ac20e96469cadb7fc92f62f1c01, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.418280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.457733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1095/2188] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:39 2022] PUT /identity/v3/projects/590007d5ba314dfb8df70c171578a238/users/4b5d5ac20e96469cadb7fc92f62f1c01/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:01:39.462415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.462748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.463183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.534191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] MFA Rules not processed for user `4b5d5ac20e96469cadb7fc92f62f1c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:39.535808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:39.673995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1094/2189] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:39.679349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.762776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:39.773066 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724a66e-2cf2-4360-b66e-51bd92564c6e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.781572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.782629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.783364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.783825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.784969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.788482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.827283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1096/2190] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:39 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:39.832566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.850811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd845a-ae9d-43bc-bf16-7ca2ac41a773 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.923963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.924585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.924870 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.925141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.925778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.927752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:39.956146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1095/2191] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:39 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:39.958825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:39.971834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e87f5-4937-4044-82a2-dea61ebb2ddd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:39.984165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:39.984165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:39.984490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:39.985097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:39.985259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:39.987005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:40.015407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1097/2192] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:40.019095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:40.029932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c3db-6298-49b5-b2da-e5c28e899da8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:40.037565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:40.038547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:40.038821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:40.039189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] PATH_INFO: `/v3/projects/c4f3efa9886e428dba2f67d229165891/users/11f1905572d54712a764063e1008850a/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:40.057081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f3efa9886e428dba2f67d229165891, user_id=11f1905572d54712a764063e1008850a, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:40.059915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:40.077312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1096/2193] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:40 2022] PUT /identity/v3/projects/c4f3efa9886e428dba2f67d229165891/users/11f1905572d54712a764063e1008850a/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:40.081537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:40.092049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9132-9acb-4752-93aa-d560428d1bb3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:40.164831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6612238-f31f-4181-a873-b266ca4193ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:40.165843 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6612238-f31f-4181-a873-b266ca4193ae admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:40.166173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6612238-f31f-4181-a873-b266ca4193ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:40.166794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6612238-f31f-4181-a873-b266ca4193ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:40.167720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6612238-f31f-4181-a873-b266ca4193ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:40.169488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6612238-f31f-4181-a873-b266ca4193ae admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:40.188207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1098/2194] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:40.190765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:40.198356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:40.214394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b38b10-6d55-4042-8958-780fcaea98a0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:40.223494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:40.224240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:40.224548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:40.224829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] PATH_INFO: `/v3/projects/c4f3efa9886e428dba2f67d229165891/users/11f1905572d54712a764063e1008850a/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:40.227091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f3efa9886e428dba2f67d229165891, user_id=11f1905572d54712a764063e1008850a, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:40.230529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:40.250939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1097/2195] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:40 2022] PUT /identity/v3/projects/c4f3efa9886e428dba2f67d229165891/users/11f1905572d54712a764063e1008850a/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:40.260058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:40.260419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:40.260680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:40.293730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] MFA Rules not processed for user `11f1905572d54712a764063e1008850a`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:40.295332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:40.376240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1099/2196] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:40.388158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:40.389900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:40.453457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b2d0-d0d3-408c-ac95-0489c4b646ab admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:40.458053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515b79f-adc6-449e-aa9b-a69547ee9ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:40.458666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515b79f-adc6-449e-aa9b-a69547ee9ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:40.458974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515b79f-adc6-449e-aa9b-a69547ee9ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:40.459247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515b79f-adc6-449e-aa9b-a69547ee9ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:40.469161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515b79f-adc6-449e-aa9b-a69547ee9ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:40.470746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515b79f-adc6-449e-aa9b-a69547ee9ec8 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:40.481122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1098/2197] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:41.055008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:41.121087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:41.136361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc173f-4bee-4356-b81b-4d9d5f6d43da None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:41.142811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:41.143600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:41.143992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:41.144163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] PATH_INFO: `/v3/users/4b5d5ac20e96469cadb7fc92f62f1c01` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:41.145343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5d5ac20e96469cadb7fc92f62f1c01)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:41.147470 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:41.199028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5d5ac20e96469cadb7fc92f62f1c01'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:41.199699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5d5ac20e96469cadb7fc92f62f1c01'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:41.208001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5d5ac20e96469cadb7fc92f62f1c01'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:41.223869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5d5ac20e96469cadb7fc92f62f1c01'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:41.224660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-0777f278-1332-4258-8122-bc22c92efd6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5d5ac20e96469cadb7fc92f62f1c01'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:41.240962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1100/2198] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:41 2022] DELETE /identity/v3/users/4b5d5ac20e96469cadb7fc92f62f1c01 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:41.246015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:41.247871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:41.249446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:41.292881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:41.295284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:41.341571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:41.341571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:41.341832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:41.370707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] MFA Rules not processed for user `9979be70f604408d9b78050e3ba52f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:41.372151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:41.439390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1099/2199] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:41.453860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:41.456097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:41.494565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1101/2200] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:01:41.538075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86cc62-8978-4e35-b16b-62c6a773f485 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:41.542670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:41.543561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:41.543896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:41.544573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:41.564431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:41.566180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:41.581370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1100/2201] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:01:42.155909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:42.159781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:42.184429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730219-e151-43cc-bcfd-1bbd980c73a2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:42.206621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:42.207697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:42.208127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:42.208535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] PATH_INFO: `/v3/projects/590007d5ba314dfb8df70c171578a238` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:42.210301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=590007d5ba314dfb8df70c171578a238)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:42.213416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:42.282115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '590007d5ba314dfb8df70c171578a238', 'request_id': 'req-816c8c0e-3f25-4400-bb83-c420572aada4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.289663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '590007d5ba314dfb8df70c171578a238', 'request_id': 'req-816c8c0e-3f25-4400-bb83-c420572aada4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.296825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '590007d5ba314dfb8df70c171578a238', 'request_id': 'req-816c8c0e-3f25-4400-bb83-c420572aada4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.308934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:42.335479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] The token cache is being invalidate because project 590007d5ba314dfb8df70c171578a238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:42.335479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.335479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.335479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1102/2202] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:42 2022] DELETE /identity/v3/projects/590007d5ba314dfb8df70c171578a238 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:42.336858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:42.347856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a94b7-a68e-4376-b43c-17ca05ada3dc service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:42.386967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:42.420835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:42.421476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:42.421760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:42.422036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] PATH_INFO: `/v3/users/11f1905572d54712a764063e1008850a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:42.423711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f1905572d54712a764063e1008850a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:42.425670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:42.483047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f1905572d54712a764063e1008850a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.484095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f1905572d54712a764063e1008850a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.485175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f1905572d54712a764063e1008850a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.496466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f1905572d54712a764063e1008850a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.511852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2bd19aec-d047-4c2f-85d8-73a776885410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f1905572d54712a764063e1008850a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:42.527983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1101/2203] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:42 2022] DELETE /identity/v3/users/11f1905572d54712a764063e1008850a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:43.015784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:43.018609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:43.042094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c8c0e-3f25-4400-bb83-c420572aada4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:43.134510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:43.135193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.135620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.136144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] PATH_INFO: `/v3/projects/c4f3efa9886e428dba2f67d229165891` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.137709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f3efa9886e428dba2f67d229165891)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:43.141128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:43.264431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f3efa9886e428dba2f67d229165891', 'request_id': 'req-624b1046-f4e9-4803-b6f4-d43cfd47de6f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:43.293800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f3efa9886e428dba2f67d229165891', 'request_id': 'req-624b1046-f4e9-4803-b6f4-d43cfd47de6f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:43.318598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f3efa9886e428dba2f67d229165891', 'request_id': 'req-624b1046-f4e9-4803-b6f4-d43cfd47de6f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:43.343558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] The token cache is being invalidate because project c4f3efa9886e428dba2f67d229165891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:43.346183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:43.348781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-624b1046-f4e9-4803-b6f4-d43cfd47de6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:43.357671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1103/2204] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:43 2022] DELETE /identity/v3/projects/c4f3efa9886e428dba2f67d229165891 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:43.362132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.363202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.363646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.400654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:43.406530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:43.537246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1102/2205] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:43.546709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.547323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.548036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.600805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:43.602084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:43.628049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1104/2206] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:43.632735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:43.647243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b112924-6a3d-462d-a824-6ca977a67ce1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:43.652852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:43.653550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.653894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.654246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.655022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:01:43.655545 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:43.657852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:43.671712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1103/2207] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:01:43.676365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:43.689584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5754120-89ba-4332-85ba-354a8e88af65 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:43.695727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:43.696317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.696600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.696976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.697616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:43.700019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:43.744849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1105/2208] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:43 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:43.749112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:43.766035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd658ac0-74cf-4955-bda2-50b91ea149f2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:43.834050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:43.857012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:43.859791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.860240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.861188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.862270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:43.878240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:43.913611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1104/2209] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:43 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:43.917424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:43.932241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecaead-749d-4ce9-9e2e-e030470a38e0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:43.953441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:43.956113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:43.957346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:43.957645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:43.959605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:43.963704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:43.998680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1106/2210] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:44.002549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.016519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cbd48-61df-44c2-8a25-b83bab44d86a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.061669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f103e-c751-4f9f-8c72-e57ef8f386c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.062301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f103e-c751-4f9f-8c72-e57ef8f386c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.062584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f103e-c751-4f9f-8c72-e57ef8f386c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.062881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f103e-c751-4f9f-8c72-e57ef8f386c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.063561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f103e-c751-4f9f-8c72-e57ef8f386c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.065569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f103e-c751-4f9f-8c72-e57ef8f386c7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.082747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.087937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1105/2211] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:44.109972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f22a6-7411-4084-99d2-418401e105b9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.129581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a9431-9f9c-4392-9cf2-89699fe0b6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.130891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a9431-9f9c-4392-9cf2-89699fe0b6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.130891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a9431-9f9c-4392-9cf2-89699fe0b6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.130891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a9431-9f9c-4392-9cf2-89699fe0b6f4 admin admin] PATH_INFO: `/v3/projects/930f2cdbe8d947a8a0247ce4a1ea9afd/users/1531c72a057e4307836f4691b781a259/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.172949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a9431-9f9c-4392-9cf2-89699fe0b6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930f2cdbe8d947a8a0247ce4a1ea9afd, user_id=1531c72a057e4307836f4691b781a259, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.177561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a9431-9f9c-4392-9cf2-89699fe0b6f4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.203069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1107/2212] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:44 2022] PUT /identity/v3/projects/930f2cdbe8d947a8a0247ce4a1ea9afd/users/1531c72a057e4307836f4691b781a259/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:44.214733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.215149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.215367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.267668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.268372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.268372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.274220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] MFA Rules not processed for user `1531c72a057e4307836f4691b781a259`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:44.275499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:44.297158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:44.299647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:44.346208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:44.378754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1106/2213] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:44.383237 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.424160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1108/2214] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:01:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:44.454663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8f0b3-2756-4ea7-be27-b024d3727280 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.459771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.460624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.460936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.461086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.461797 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.463858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.491275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.493011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:44.498153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1107/2215] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:44 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:44.504088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.588473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:44.596951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac62392-6b8f-4776-acd9-03d5a538bbc2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.621894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869cde5-9345-4e7e-b502-fe99a6aa88eb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.628496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.629118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.629697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.629820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.630746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.633068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.666385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1108/2216] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:44 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:01:44.668228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.675924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.676926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.677340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.677698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.679222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f3e16-3d6b-4a5d-87fb-9babcafcff5e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.696435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.697056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.697330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.697598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.698399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.700310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.719170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1109/2217] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:44.721517 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.732381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d0b2c-1054-44d7-944d-e2ac817bab33 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.745954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.747260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.747655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.748039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] PATH_INFO: `/v3/projects/913e6b31a2af45409c2322e12bdae99a/users/5e21c8fbf57648c5967492a1a71d92fa/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.755069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.757043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.771430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1109/2218] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:01:44.773889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913e6b31a2af45409c2322e12bdae99a, user_id=5e21c8fbf57648c5967492a1a71d92fa, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.777262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.805679 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1110/2219] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:44 2022] PUT /identity/v3/projects/913e6b31a2af45409c2322e12bdae99a/users/5e21c8fbf57648c5967492a1a71d92fa/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:44.807549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:44.852225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aefca-542f-400e-82f2-070523039e21 service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:44.865569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:44.969524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763eb82a-361b-4c6f-8557-97f28209f992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:44.972169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763eb82a-361b-4c6f-8557-97f28209f992 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:44.972169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763eb82a-361b-4c6f-8557-97f28209f992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:44.972169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763eb82a-361b-4c6f-8557-97f28209f992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:44.972447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763eb82a-361b-4c6f-8557-97f28209f992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:44.974037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763eb82a-361b-4c6f-8557-97f28209f992 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:44.997746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1110/2220] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:45.002739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:45.013744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acf13e-bf72-447e-958b-672ca9407ff5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:45.030197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:45.030973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:45.031259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:45.031538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] PATH_INFO: `/v3/projects/913e6b31a2af45409c2322e12bdae99a/users/5e21c8fbf57648c5967492a1a71d92fa/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:45.034285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913e6b31a2af45409c2322e12bdae99a, user_id=5e21c8fbf57648c5967492a1a71d92fa, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:45.037415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:45.061062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1111/2221] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:45 2022] PUT /identity/v3/projects/913e6b31a2af45409c2322e12bdae99a/users/5e21c8fbf57648c5967492a1a71d92fa/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:45.072593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:45.073936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:45.073936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:45.115608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] MFA Rules not processed for user `5e21c8fbf57648c5967492a1a71d92fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:45.116955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:45.268298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1111/2222] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:45.281683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:45.286075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:45.429796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda236-fbf8-4810-85a5-3ce2d598a653 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:45.434582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef48618-a9f1-4969-b110-df5526a88720 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:45.435464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48618-a9f1-4969-b110-df5526a88720 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:45.436103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48618-a9f1-4969-b110-df5526a88720 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:45.439151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48618-a9f1-4969-b110-df5526a88720 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:45.457361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48618-a9f1-4969-b110-df5526a88720 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:45.459376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48618-a9f1-4969-b110-df5526a88720 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:45.471852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1112/2223] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:48.298703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:48.469872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b942a5-5ca3-46d4-8a15-42716e79e0fc None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:48.477452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:48.478710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:48.479405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:48.479834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] PATH_INFO: `/v3/users/1531c72a057e4307836f4691b781a259` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:48.481290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1531c72a057e4307836f4691b781a259)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:48.483948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:48.569403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:48.588427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1531c72a057e4307836f4691b781a259'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:48.589192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1531c72a057e4307836f4691b781a259'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:48.600709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1531c72a057e4307836f4691b781a259'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:48.638091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1531c72a057e4307836f4691b781a259'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:48.638870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1531c72a057e4307836f4691b781a259'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:48.659076 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1112/2224] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:48 2022] DELETE /identity/v3/users/1531c72a057e4307836f4691b781a259 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:48.665891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:48.667193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:48.667656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:48.693560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:48.695817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:48.810775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1113/2225] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:48.822427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:48.824834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:48.952893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbcf0c-d493-46cd-a835-893bde8b4d2a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:48.958081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:48.959333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:48.959839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:48.960113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:48.991459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:48.992909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:49.009744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1113/2226] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:49.500426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:49.502790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:49.531824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3c8a8-aa34-4018-82c6-65c02ff62f8c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:49.542956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:49.543626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:49.543924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:49.544182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] PATH_INFO: `/v3/projects/930f2cdbe8d947a8a0247ce4a1ea9afd` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:49.546715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=930f2cdbe8d947a8a0247ce4a1ea9afd)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:49.549116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:49.581764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:49.632685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930f2cdbe8d947a8a0247ce4a1ea9afd', 'request_id': 'req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.640126 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930f2cdbe8d947a8a0247ce4a1ea9afd', 'request_id': 'req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.653388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930f2cdbe8d947a8a0247ce4a1ea9afd', 'request_id': 'req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.667207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] The token cache is being invalidate because project 930f2cdbe8d947a8a0247ce4a1ea9afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:49.667891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.668163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.670139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1114/2227] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:49 2022] DELETE /identity/v3/projects/930f2cdbe8d947a8a0247ce4a1ea9afd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:49.674513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:49.694051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884770-d573-4060-ade5-c7e6cca95d2b service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:49.848122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:49.849392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:49.849674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:49.850031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] PATH_INFO: `/v3/users/5e21c8fbf57648c5967492a1a71d92fa` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:49.851954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e21c8fbf57648c5967492a1a71d92fa)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:49.855308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:49.952177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e21c8fbf57648c5967492a1a71d92fa'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.952990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e21c8fbf57648c5967492a1a71d92fa'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.963409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e21c8fbf57648c5967492a1a71d92fa'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.966274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:49.985749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e21c8fbf57648c5967492a1a71d92fa'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:49.986505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-3c83584a-c10c-4863-8915-ebaaa2915791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e21c8fbf57648c5967492a1a71d92fa'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:50.016119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1114/2228] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:01:49 2022] DELETE /identity/v3/users/5e21c8fbf57648c5967492a1a71d92fa => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:50.955425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:50.956917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:50.995572 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7f28d-1c66-4cfb-82c3-2ad4104ae25d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:51.096300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:51.097278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.097660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.098032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] PATH_INFO: `/v3/projects/913e6b31a2af45409c2322e12bdae99a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.099608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913e6b31a2af45409c2322e12bdae99a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:51.101648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:51.243676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913e6b31a2af45409c2322e12bdae99a', 'request_id': 'req-64d75e3d-5d52-418c-bfd0-eff262831531'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:51.255194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913e6b31a2af45409c2322e12bdae99a', 'request_id': 'req-64d75e3d-5d52-418c-bfd0-eff262831531'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:51.265676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913e6b31a2af45409c2322e12bdae99a', 'request_id': 'req-64d75e3d-5d52-418c-bfd0-eff262831531'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:51.281671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] The token cache is being invalidate because project 913e6b31a2af45409c2322e12bdae99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:01:51.282064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:51.282822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-64d75e3d-5d52-418c-bfd0-eff262831531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:01:51.284866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1115/2229] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:01:50 2022] DELETE /identity/v3/projects/913e6b31a2af45409c2322e12bdae99a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:51.291074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.291634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.291634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.315632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:51.317002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:51.464645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1115/2230] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:51.466482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.466638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.466914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.508849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:51.510097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:51.535591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1116/2231] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:51.539634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:51.565343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d2c5d-1839-4240-82ff-47f19b8d084d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:51.578618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:51.580827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.581327 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.583103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.584053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:01:51.585061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:51.588326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:51.611789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1116/2232] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:01:51.616696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:51.628676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b4c0d-2907-4ab7-86ad-f82e5c166c09 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:51.634396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:51.635054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.635342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.635656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.636373 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:51.638423 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:51.686301 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1117/2233] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:51 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:01:51.691835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:51.723171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225127c-3b16-4638-a1ba-7b234b4a7738 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:51.827594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:51.828255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.828585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.828850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.829727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:51.843593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:51.877640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1117/2234] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:51 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:51.881839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:51.890198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:51.902644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028821fd-d7f5-48ab-9b3f-b3e30bcc810f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:51.913248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:51.913877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.914159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.914430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.915971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:51.920457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:51.939073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1118/2235] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:51.943447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:51.958239 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468162-be8d-4e7d-8a7e-f3561dcdd8d3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:51.975319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b5e82-d5fe-4d57-866f-6edf8d0ac5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:51.976023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5e82-d5fe-4d57-866f-6edf8d0ac5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:51.976404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5e82-d5fe-4d57-866f-6edf8d0ac5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:51.976763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b5e82-d5fe-4d57-866f-6edf8d0ac5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:51.977680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b5e82-d5fe-4d57-866f-6edf8d0ac5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:51.982874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b5e82-d5fe-4d57-866f-6edf8d0ac5d5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:52.026472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:52.040323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1118/2236] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:52.054396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43cc7c-f308-40c4-abf9-1182000d2bb9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:52.069454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:52.070475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:52.070750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:52.071069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] PATH_INFO: `/v3/projects/ca00dc6e8e154184a9eb6dfffe4faf36/users/39df385f05804b0a8b8babf10398b906/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:52.109100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca00dc6e8e154184a9eb6dfffe4faf36, user_id=39df385f05804b0a8b8babf10398b906, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:52.112176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:52.141146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1119/2237] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:52 2022] PUT /identity/v3/projects/ca00dc6e8e154184a9eb6dfffe4faf36/users/39df385f05804b0a8b8babf10398b906/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:01:52.148599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769104-6e47-430e-94af-d5d31045b348 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:52.149028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769104-6e47-430e-94af-d5d31045b348 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:52.149467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769104-6e47-430e-94af-d5d31045b348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:52.190084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-67769104-6e47-430e-94af-d5d31045b348 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:52.192353 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-67769104-6e47-430e-94af-d5d31045b348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:52.244732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:52.272703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1119/2238] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:52.311549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:52.313768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:52.426321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37800-0cd7-491f-a76e-4dad89a0e697 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:52.430610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:52.431312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:52.431617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:52.432031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:52.443540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:52.447277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:52.459304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1120/2239] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:53.453647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:53.454272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:53.454536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:53.511572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] MFA Rules not processed for user `39df385f05804b0a8b8babf10398b906`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:53.512848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:53.688937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1120/2240] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:53.694080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:53.717093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d9b5c-8d47-4b7e-867d-b32188dd0997 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:53.732582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:53.743654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:53.744250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:53.744606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:53.745759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:53.753048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:53.817859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1121/2241] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:01:53 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:01:53.821136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:53.835712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe2b4e-5938-4420-9dc7-445256ff466f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:53.994722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:53.995438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:53.995726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:53.996023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:53.996966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:53.999067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:54.026658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1121/2242] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:01:53 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:01:54.032929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:54.045776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e807b-ef99-474a-9b44-f2ad342ec12d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:54.052358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:54.053224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:54.053723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:54.054070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:54.054799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:54.056521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:54.070438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1122/2243] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:01:54.075841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:54.088836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f06e1-1113-41b5-8e24-5a67a2e5d0c1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:54.094502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:54.096273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:54.096583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:54.096879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] PATH_INFO: `/v3/projects/b92eb5d9dc284a0685860963535298fa/users/8576207341c54a35bfba6f94a7f11c62/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:54.116645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b92eb5d9dc284a0685860963535298fa, user_id=8576207341c54a35bfba6f94a7f11c62, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:54.119488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:54.138448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1122/2244] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:54 2022] PUT /identity/v3/projects/b92eb5d9dc284a0685860963535298fa/users/8576207341c54a35bfba6f94a7f11c62/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:54.139703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:54.159603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff15aa-53f8-4302-ba7d-c374d119be12 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:54.241355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7083f-295d-4241-b293-f490d1f3c73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:54.242167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7083f-295d-4241-b293-f490d1f3c73c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:54.242505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7083f-295d-4241-b293-f490d1f3c73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:54.242920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7083f-295d-4241-b293-f490d1f3c73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:54.244554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7083f-295d-4241-b293-f490d1f3c73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:54.246460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7083f-295d-4241-b293-f490d1f3c73c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:54.258622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1123/2245] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:01:54.263592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:54.277196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a072b-b6ce-4bc2-9876-28422f12072e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:54.282647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:54.283511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:54.283876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:54.284359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] PATH_INFO: `/v3/projects/b92eb5d9dc284a0685860963535298fa/users/8576207341c54a35bfba6f94a7f11c62/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:54.286404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b92eb5d9dc284a0685860963535298fa, user_id=8576207341c54a35bfba6f94a7f11c62, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:54.289383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:54.302273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1123/2246] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:01:54 2022] PUT /identity/v3/projects/b92eb5d9dc284a0685860963535298fa/users/8576207341c54a35bfba6f94a7f11c62/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:01:55.520787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:55.521441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:55.522161 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:55.562560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] MFA Rules not processed for user `8576207341c54a35bfba6f94a7f11c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:55.564137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:55.604597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:55.678854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1124/2247] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:01:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:55.691751 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:55.693791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:55.731053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:55.771842 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ef089-c8df-4361-b59c-4fabfc683f22 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:55.777136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f7663-7c1f-4c01-b4df-c6f6f731d846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:55.778664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f7663-7c1f-4c01-b4df-c6f6f731d846 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:55.779365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f7663-7c1f-4c01-b4df-c6f6f731d846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:55.780339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f7663-7c1f-4c01-b4df-c6f6f731d846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:55.863620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f7663-7c1f-4c01-b4df-c6f6f731d846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:55.865437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f7663-7c1f-4c01-b4df-c6f6f731d846 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:55.890612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1124/2248] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:57.448042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:57.450514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:57.465827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f26a47-9b18-43cc-ad4a-7e05b6cdfba0 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:57.471026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c48f65-6bfc-4f36-80f7-c34812437c85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:57.472952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c48f65-6bfc-4f36-80f7-c34812437c85 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:57.473252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c48f65-6bfc-4f36-80f7-c34812437c85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:57.475101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c48f65-6bfc-4f36-80f7-c34812437c85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:57.489827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c48f65-6bfc-4f36-80f7-c34812437c85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:57.491709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c48f65-6bfc-4f36-80f7-c34812437c85 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:57.516314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1125/2249] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:05.677515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab02a-c8f3-4a87-b457-037b03f6a265 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:05.678154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab02a-c8f3-4a87-b457-037b03f6a265 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:05.678651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ab02a-c8f3-4a87-b457-037b03f6a265 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:05.680504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1125/2250] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:02:05.687022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146bbbc-6ab9-4f9c-98c1-6204aeca1f0a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:05.688157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146bbbc-6ab9-4f9c-98c1-6204aeca1f0a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:05.689032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146bbbc-6ab9-4f9c-98c1-6204aeca1f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:05.724543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4146bbbc-6ab9-4f9c-98c1-6204aeca1f0a None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:05.726083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4146bbbc-6ab9-4f9c-98c1-6204aeca1f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:05.871754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=37 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:05.888398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1126/2251] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:02:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:02:05.902293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ab02a-c8f3-4a87-b457-037b03f6a265 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:05.904236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bb7ab02a-c8f3-4a87-b457-037b03f6a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:05.991883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ab02a-c8f3-4a87-b457-037b03f6a265 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:06.009162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fed34-4bc0-40f0-a69b-20c081b2e806 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:06.010165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fed34-4bc0-40f0-a69b-20c081b2e806 service glance] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.010735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fed34-4bc0-40f0-a69b-20c081b2e806 service glance] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.011118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fed34-4bc0-40f0-a69b-20c081b2e806 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.023859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fed34-4bc0-40f0-a69b-20c081b2e806 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:06.026764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fed34-4bc0-40f0-a69b-20c081b2e806 service glance] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:06.040690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1126/2252] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:06.135134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f09666-9d31-453f-90db-6b5a8656758b None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.135429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f09666-9d31-453f-90db-6b5a8656758b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.135706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f09666-9d31-453f-90db-6b5a8656758b None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.137075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1127/2253] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:02:06.143064 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b068d-ab13-4708-9df9-55fef81cdbdd None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.143413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b068d-ab13-4708-9df9-55fef81cdbdd None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.143693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b068d-ab13-4708-9df9-55fef81cdbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.165450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-688b068d-ab13-4708-9df9-55fef81cdbdd None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:06.166674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-688b068d-ab13-4708-9df9-55fef81cdbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:06.240528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1127/2254] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:02:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:06.254302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f09666-9d31-453f-90db-6b5a8656758b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:06.258164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-77f09666-9d31-453f-90db-6b5a8656758b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:06.273343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f09666-9d31-453f-90db-6b5a8656758b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:06.277102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc6817-8234-4033-8268-feaff802c62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:06.277755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6817-8234-4033-8268-feaff802c62f service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.277921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6817-8234-4033-8268-feaff802c62f service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.278174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6817-8234-4033-8268-feaff802c62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.288894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc6817-8234-4033-8268-feaff802c62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:06.290728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc6817-8234-4033-8268-feaff802c62f service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:06.300455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1128/2255] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:06.401813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b6389-5599-4109-851b-949bc32b3009 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.402128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b6389-5599-4109-851b-949bc32b3009 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.402406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b6389-5599-4109-851b-949bc32b3009 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.403796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1128/2256] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:02:06.410201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.410492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.410769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.431113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:06.432336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:06.442253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1129/2257] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:02:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:06.453032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b6389-5599-4109-851b-949bc32b3009 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:06.464875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b6389-5599-4109-851b-949bc32b3009 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:06.468436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220e267-6aed-48be-ab1b-73a81bee0716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:06.469027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220e267-6aed-48be-ab1b-73a81bee0716 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:06.469304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220e267-6aed-48be-ab1b-73a81bee0716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:06.469590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220e267-6aed-48be-ab1b-73a81bee0716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:06.479870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220e267-6aed-48be-ab1b-73a81bee0716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:06.481300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220e267-6aed-48be-ab1b-73a81bee0716 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:06.492937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1129/2258] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:07.519628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:07.529365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d8143-51bc-4f42-94b9-04e90bf3364b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:07.533901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:07.534518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:07.534856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:07.535143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:07.538154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:07.615460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:07.616962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8ef03-e8f9-419c-9757-908aa27e3859 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:07.630181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1130/2259] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:02:08.269694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455bc0-03ca-46e9-aa43-ece1784297a7 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:08.270228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455bc0-03ca-46e9-aa43-ece1784297a7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:08.270228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65455bc0-03ca-46e9-aa43-ece1784297a7 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:08.271848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1130/2260] 10.222.0.72 () {54 vars in 961 bytes} [Mon Sep 26 18:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:02:08.277101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d00e-ce6a-4a57-bc3e-4b91ac9c77aa None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:08.277270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d00e-ce6a-4a57-bc3e-4b91ac9c77aa None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:08.277570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31d00e-ce6a-4a57-bc3e-4b91ac9c77aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:08.298393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8e31d00e-ce6a-4a57-bc3e-4b91ac9c77aa None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:08.299671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8e31d00e-ce6a-4a57-bc3e-4b91ac9c77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:08.309151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:08.369333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1131/2261] 10.222.0.72 () {58 vars in 1058 bytes} [Mon Sep 26 18:02:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:02:08.377651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455bc0-03ca-46e9-aa43-ece1784297a7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:08.392419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65455bc0-03ca-46e9-aa43-ece1784297a7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:08.398698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:08.399403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:08.399686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:08.400018 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:08.401892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:08.415487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:08.417122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:08.428074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1131/2262] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:11.148168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed84a09-200d-479c-b2b1-aefec7893203 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:11.149136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed84a09-200d-479c-b2b1-aefec7893203 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:11.149437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed84a09-200d-479c-b2b1-aefec7893203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:11.172487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-eed84a09-200d-479c-b2b1-aefec7893203 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:11.174410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-eed84a09-200d-479c-b2b1-aefec7893203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:11.243004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1132/2263] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:11.256431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:11.258582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:11.353625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d425d7-3d37-4b0c-83f2-b1ed8e48f6e0 service placement] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:11.363242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bca02-30cf-4a20-8efa-2ed17870dbc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:11.364278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bca02-30cf-4a20-8efa-2ed17870dbc4 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:11.364775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bca02-30cf-4a20-8efa-2ed17870dbc4 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:11.365270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bca02-30cf-4a20-8efa-2ed17870dbc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:11.378094 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bca02-30cf-4a20-8efa-2ed17870dbc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:11.379784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bca02-30cf-4a20-8efa-2ed17870dbc4 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:11.388503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1132/2264] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:02:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:02:11.510897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01823bb1-fb4c-4919-a351-e768ff1771c7 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:11.511333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01823bb1-fb4c-4919-a351-e768ff1771c7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:11.511506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01823bb1-fb4c-4919-a351-e768ff1771c7 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:11.513504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1133/2265] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:02:11.560002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:11.560284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:11.560563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:11.613536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] MFA Rules not processed for user `92937af60f80421094f4bdf6c7620283`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:11.616250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:11.630175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1133/2266] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:02:11.662686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01823bb1-fb4c-4919-a351-e768ff1771c7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:11.679481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01823bb1-fb4c-4919-a351-e768ff1771c7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:11.685558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:11.686652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:11.687028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:11.687549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:11.700211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:11.702264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:11.715149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1134/2267] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:12.282198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:12.294489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:12.374433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fad4f-ac2b-4eb9-a9c5-ac0f1b911002 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:12.381068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:12.381742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:12.382106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:12.382465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] PATH_INFO: `/v3/users/39df385f05804b0a8b8babf10398b906` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:12.383738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39df385f05804b0a8b8babf10398b906)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:12.385693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:12.440850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39df385f05804b0a8b8babf10398b906'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.441407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39df385f05804b0a8b8babf10398b906'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.451271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39df385f05804b0a8b8babf10398b906'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.464256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39df385f05804b0a8b8babf10398b906'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.464980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39df385f05804b0a8b8babf10398b906'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.480367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1134/2268] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:02:12 2022] DELETE /identity/v3/users/39df385f05804b0a8b8babf10398b906 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:12.703425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:12.722958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b920e6d-1b79-4b7b-b568-3c6327ab8a97 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:12.811542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:12.812192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:12.812470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:12.812946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] PATH_INFO: `/v3/projects/ca00dc6e8e154184a9eb6dfffe4faf36` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:12.814752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca00dc6e8e154184a9eb6dfffe4faf36)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:12.816900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:12.883768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:12.888441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca00dc6e8e154184a9eb6dfffe4faf36', 'request_id': 'req-81de4f9a-a7e0-4504-b165-36428c7c972e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.899026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca00dc6e8e154184a9eb6dfffe4faf36', 'request_id': 'req-81de4f9a-a7e0-4504-b165-36428c7c972e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.914094 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca00dc6e8e154184a9eb6dfffe4faf36', 'request_id': 'req-81de4f9a-a7e0-4504-b165-36428c7c972e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.929696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] The token cache is being invalidate because project ca00dc6e8e154184a9eb6dfffe4faf36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:02:12.935270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.936041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:12.937900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1135/2269] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:02:12 2022] DELETE /identity/v3/projects/ca00dc6e8e154184a9eb6dfffe4faf36 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:12.942736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:12.944307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:12.960851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b8771-cd6f-41ee-b5eb-cf3df5c2c602 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:13.037169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:13.037823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:13.038107 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:13.038383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] PATH_INFO: `/v3/users/8576207341c54a35bfba6f94a7f11c62` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:13.039557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8576207341c54a35bfba6f94a7f11c62)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:13.041483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:13.081831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:13.100376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8576207341c54a35bfba6f94a7f11c62'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.101059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8576207341c54a35bfba6f94a7f11c62'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.109057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8576207341c54a35bfba6f94a7f11c62'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.123978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8576207341c54a35bfba6f94a7f11c62'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.124795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-d8b0c956-bf9b-42b0-bd9c-cac8e17df8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8576207341c54a35bfba6f94a7f11c62'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.143685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1135/2270] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:02:12 2022] DELETE /identity/v3/users/8576207341c54a35bfba6f94a7f11c62 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:13.368230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:13.373549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:13.388209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de4f9a-a7e0-4504-b165-36428c7c972e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:13.476997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:13.477645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:13.477933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:13.478207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] PATH_INFO: `/v3/projects/b92eb5d9dc284a0685860963535298fa` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:13.479557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b92eb5d9dc284a0685860963535298fa)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:13.481510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:13.545334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b92eb5d9dc284a0685860963535298fa', 'request_id': 'req-153f3577-0181-4d1c-9816-33c31501e42a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.552581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b92eb5d9dc284a0685860963535298fa', 'request_id': 'req-153f3577-0181-4d1c-9816-33c31501e42a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.561894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b92eb5d9dc284a0685860963535298fa', 'request_id': 'req-153f3577-0181-4d1c-9816-33c31501e42a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.575156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] The token cache is being invalidate because project b92eb5d9dc284a0685860963535298fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:02:13.575540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.576156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-153f3577-0181-4d1c-9816-33c31501e42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:13.577819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1136/2271] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:02:13 2022] DELETE /identity/v3/projects/b92eb5d9dc284a0685860963535298fa => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:13.584220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:13.584779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:13.584779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:13.605896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:13.607237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:13.677362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1136/2272] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:02:13.685088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:13.685519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:13.685811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:13.719365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:13.720659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:13.751025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1137/2273] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:13.753112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:13.764159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e148975-0a19-46fb-b739-758e029e4eea None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:13.778363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:13.779114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:13.779501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:13.779834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:13.780582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:02:13.780950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:13.784000 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:13.819120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:13.838395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1137/2274] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:02:13.841825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c32a7-53de-43bf-ac23-d981118a14fe None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:13.847728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:13.848499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:13.848785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:13.849066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:13.849707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:13.851785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:13.924189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1138/2275] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:02:13 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:13.928780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:13.940632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40eed6-3753-4f3e-b3e4-3e3b4d37c4b7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.065146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.065146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.065146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.065146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.065638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.066823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:14.097750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1138/2276] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:02:13 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:14.099124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:14.118977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20955d1b-69ec-433e-8715-8f96b918bd0f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.125546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.134888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.134888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.134888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.134888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.134888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:14.145118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1139/2277] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:14.148733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:14.159517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b5c1d-9101-449b-a506-6359ab37ebcd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.164785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe012a-17c6-4409-ba75-69fadfe51e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.165412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe012a-17c6-4409-ba75-69fadfe51e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.165698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe012a-17c6-4409-ba75-69fadfe51e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.165975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe012a-17c6-4409-ba75-69fadfe51e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.166634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe012a-17c6-4409-ba75-69fadfe51e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.168437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe012a-17c6-4409-ba75-69fadfe51e02 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:14.182871 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1139/2278] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:14.187082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:14.200649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bd462-2564-4925-81c6-bf2a700532bf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.206626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.207328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.207614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.207892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] PATH_INFO: `/v3/projects/b4a6ac5a5f3d4c56b3722ffcae7eebdb/users/ba01ef4d092040bf93049a3996a10ca3/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.231638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a6ac5a5f3d4c56b3722ffcae7eebdb, user_id=ba01ef4d092040bf93049a3996a10ca3, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.234354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:14.244787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:14.252470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1140/2279] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:02:14 2022] PUT /identity/v3/projects/b4a6ac5a5f3d4c56b3722ffcae7eebdb/users/ba01ef4d092040bf93049a3996a10ca3/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:02:14.264580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.264927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.265133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.290676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:14.294023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] MFA Rules not processed for user `ba01ef4d092040bf93049a3996a10ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:14.296412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:14.426178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1140/2280] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:14.460415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:14.463059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:14.530194 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25231d75-db93-40cb-bdfb-7635e3cf6a14 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.534910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcfa17-18be-44eb-a014-87b53a474399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.535537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfa17-18be-44eb-a014-87b53a474399 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.535814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfa17-18be-44eb-a014-87b53a474399 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.536090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcfa17-18be-44eb-a014-87b53a474399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.546442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfa17-18be-44eb-a014-87b53a474399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.549286 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcfa17-18be-44eb-a014-87b53a474399 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:14.563095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1141/2281] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:14.585002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:14.656150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96a4a-4e15-44e8-83e7-096a909aa04b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.661898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.662548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.662847 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.663138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] PATH_INFO: `/v3/users/ba01ef4d092040bf93049a3996a10ca3` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.664337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba01ef4d092040bf93049a3996a10ca3)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.666369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:14.723595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba01ef4d092040bf93049a3996a10ca3'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:14.724252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba01ef4d092040bf93049a3996a10ca3'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:14.733021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba01ef4d092040bf93049a3996a10ca3'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:14.746458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba01ef4d092040bf93049a3996a10ca3'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:14.753041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba01ef4d092040bf93049a3996a10ca3'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:14.772859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1141/2282] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:02:14 2022] DELETE /identity/v3/users/ba01ef4d092040bf93049a3996a10ca3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:14.777201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.777754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.777754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.799439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:14.800770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:14.870103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1142/2283] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:14.880506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:14.882111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:14.918754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:14.959460 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32980e-8938-438b-a210-a1454689e11c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:14.963285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6aabb-6f21-4709-96a4-ed1e9dcc6a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:14.964028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6aabb-6f21-4709-96a4-ed1e9dcc6a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:14.964312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6aabb-6f21-4709-96a4-ed1e9dcc6a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:14.964584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6aabb-6f21-4709-96a4-ed1e9dcc6a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:14.975239 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6aabb-6f21-4709-96a4-ed1e9dcc6a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:14.976685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6aabb-6f21-4709-96a4-ed1e9dcc6a09 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:15.001664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1142/2284] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:15.455367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:15.457500 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:15.476775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b995ef4-b445-4bc3-b281-457435b1a7d3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:15.483753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:15.485799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.486363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:15.487132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] PATH_INFO: `/v3/projects/b4a6ac5a5f3d4c56b3722ffcae7eebdb` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:15.489452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a6ac5a5f3d4c56b3722ffcae7eebdb)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:15.491958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:15.554966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a6ac5a5f3d4c56b3722ffcae7eebdb', 'request_id': 'req-8814c17d-8844-416f-ab94-24aad1f3d034'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:15.561392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a6ac5a5f3d4c56b3722ffcae7eebdb', 'request_id': 'req-8814c17d-8844-416f-ab94-24aad1f3d034'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:15.570741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a6ac5a5f3d4c56b3722ffcae7eebdb', 'request_id': 'req-8814c17d-8844-416f-ab94-24aad1f3d034'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:15.593763 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] The token cache is being invalidate because project b4a6ac5a5f3d4c56b3722ffcae7eebdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:02:15.594151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:15.595479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8814c17d-8844-416f-ab94-24aad1f3d034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:15.597722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1143/2285] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:02:15 2022] DELETE /identity/v3/projects/b4a6ac5a5f3d4c56b3722ffcae7eebdb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:15.602685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.603296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:15.603296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:15.636831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:15.638097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:15.712933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1143/2286] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:15.721067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.721351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:15.721628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:15.742864 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:15.744217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:15.758607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:15.759249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1144/2287] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:15.769124 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142818-85dd-40fe-9924-2869a8cbb08c None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:15.773796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:15.774415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.774702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:15.775015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:15.775677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:02:15.775976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:15.777577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:15.791258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1144/2288] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:02:15.795726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:15.806733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f5ab7-eedb-41bf-9543-6e433b11d403 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:15.816117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:15.816773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.817055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:15.817343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:15.817984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:15.819988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:15.837404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:15.855055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1145/2289] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:02:15 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:15.856207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:15.869589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2eb3b-8954-4730-9a64-4437966c2492 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:15.934680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:15.935330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.935612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:15.935884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:15.936511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:15.939306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:15.970609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1145/2290] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:02:15 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:15.971694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:15.988887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d5305-9d64-4710-87fc-00ff7bcbc172 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:15.996998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:15.996998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:15.996998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:16.002026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:16.002829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:16.004519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:16.023527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1146/2291] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:02:16.027334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:16.040851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459220fd-6993-4bdc-8143-a16aae3e4bae admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:16.045790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e68f3a-6db3-490b-9d75-39befac64006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:16.046411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e68f3a-6db3-490b-9d75-39befac64006 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:16.046708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e68f3a-6db3-490b-9d75-39befac64006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:16.046992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e68f3a-6db3-490b-9d75-39befac64006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:16.047795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e68f3a-6db3-490b-9d75-39befac64006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:16.050114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e68f3a-6db3-490b-9d75-39befac64006 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:16.063463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1146/2292] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:16.066533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:16.077218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cbd6b-9a05-41c8-af90-5cf650a8754c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:16.082554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:16.083262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:16.083546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:16.083822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] PATH_INFO: `/v3/projects/305855c41e574cb08475226c4ffe7795/users/65ba169876444751addc6b581263772d/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:16.101600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305855c41e574cb08475226c4ffe7795, user_id=65ba169876444751addc6b581263772d, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:16.104446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:16.120537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1147/2293] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:02:16 2022] PUT /identity/v3/projects/305855c41e574cb08475226c4ffe7795/users/65ba169876444751addc6b581263772d/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:16.131321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:16.131714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:16.132100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:16.164262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] MFA Rules not processed for user `65ba169876444751addc6b581263772d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:16.165538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:16.187390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:16.241971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1147/2294] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:16.253831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:16.255669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:16.335344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623ec21-0d27-4d2e-9e25-ea810e9085c2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:16.343658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91ab8-e1dd-499c-8cfc-0ae46712375b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:16.344310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91ab8-e1dd-499c-8cfc-0ae46712375b service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:16.344622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91ab8-e1dd-499c-8cfc-0ae46712375b service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:16.344897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91ab8-e1dd-499c-8cfc-0ae46712375b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:16.357753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c91ab8-e1dd-499c-8cfc-0ae46712375b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:16.359424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c91ab8-e1dd-499c-8cfc-0ae46712375b service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:16.369954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1148/2295] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:26.238940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=18 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:26.365008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:37.345500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:37.407700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb90e7-c3c8-4619-a5c9-af2f14d5651f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:37.413184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:37.413820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:37.414102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:37.414380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] PATH_INFO: `/v3/users/65ba169876444751addc6b581263772d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:37.415520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ba169876444751addc6b581263772d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:37.417432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:37.469747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ba169876444751addc6b581263772d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:37.470413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ba169876444751addc6b581263772d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:37.478224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ba169876444751addc6b581263772d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:37.490954 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ba169876444751addc6b581263772d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:37.491619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ba169876444751addc6b581263772d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:37.505955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1148/2296] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:02:37 2022] DELETE /identity/v3/users/65ba169876444751addc6b581263772d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:37.510349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:37.510816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:37.511010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:37.530223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:37.531473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:37.590760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1149/2297] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:37.601219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:37.602629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:37.661098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d76ba-75ad-4646-b0dd-ba8c361dd220 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:37.664972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb81ba-6f72-4629-9a64-02e494c6c334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:37.665588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb81ba-6f72-4629-9a64-02e494c6c334 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:37.665897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb81ba-6f72-4629-9a64-02e494c6c334 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:37.666173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb81ba-6f72-4629-9a64-02e494c6c334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:37.676204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb81ba-6f72-4629-9a64-02e494c6c334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:37.677625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb81ba-6f72-4629-9a64-02e494c6c334 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:37.691091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1149/2298] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:38.271555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.272989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:38.285463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab3bb-0dbc-46b9-9713-198fdca33443 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.290159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.290799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.291127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.291405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] PATH_INFO: `/v3/projects/305855c41e574cb08475226c4ffe7795` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.292516 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305855c41e574cb08475226c4ffe7795)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.294464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.346743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305855c41e574cb08475226c4ffe7795', 'request_id': 'req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:38.354829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305855c41e574cb08475226c4ffe7795', 'request_id': 'req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:38.364726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305855c41e574cb08475226c4ffe7795', 'request_id': 'req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:38.375773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] The token cache is being invalidate because project 305855c41e574cb08475226c4ffe7795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:02:38.376060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:38.376694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-7fa130b3-c61e-4c6b-b95c-672b3c5a1156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:02:38.379698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1150/2299] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:02:38 2022] DELETE /identity/v3/projects/305855c41e574cb08475226c4ffe7795 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:38.385452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.385966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.386289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.410019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:38.411261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:38.429443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:38.477955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1150/2300] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:38.488261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307d558-8654-4041-b554-3d689cee8bff None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.488549 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307d558-8654-4041-b554-3d689cee8bff None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.488820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307d558-8654-4041-b554-3d689cee8bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.509466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2307d558-8654-4041-b554-3d689cee8bff None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:38.510669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2307d558-8654-4041-b554-3d689cee8bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:38.518974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1151/2301] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:38.523185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.533717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206bbb1-bb05-42ed-a9d2-ed0c10062bd6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.538962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.539578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.539857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.540128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.540774 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:02:38.541073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.542653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.559693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1151/2302] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:02:38.561048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307d558-8654-4041-b554-3d689cee8bff None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.572010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307d558-8654-4041-b554-3d689cee8bff None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.577937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.578544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.578820 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.579122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.579761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.582779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.612368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1152/2303] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:02:38 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:38.616645 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.627003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b70541-6708-48a2-8b7c-c744e908f2d4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.683717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.684320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.684609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.684878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.685516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.687746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.709602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1152/2304] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:02:38 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:38.713727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.724838 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e518c-84e0-47f0-a3d9-44a9bb0fbf6a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.729401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.729986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.730267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.730536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.731205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.732805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.746240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1153/2305] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:38.748588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.759444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece0b4d-e743-4833-b5b2-7b088820cc3f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.764109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151e5ab-6428-4c8f-b701-a420f4d5e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.764706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151e5ab-6428-4c8f-b701-a420f4d5e0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.764989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151e5ab-6428-4c8f-b701-a420f4d5e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.765267 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151e5ab-6428-4c8f-b701-a420f4d5e0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.765911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151e5ab-6428-4c8f-b701-a420f4d5e0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.767738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151e5ab-6428-4c8f-b701-a420f4d5e0e9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.779288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1153/2306] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:38.783663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.793614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6194c-d3ee-47a1-83e8-b2786bbfb44e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:38.798259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:38.798930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.799352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.799720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] PATH_INFO: `/v3/projects/7fc64ff05f7240b3a9cb0f191195b8e3/users/13d1363fdfcf4d889f1ab927310e0374/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.820725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc64ff05f7240b3a9cb0f191195b8e3, user_id=13d1363fdfcf4d889f1ab927310e0374, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:38.824013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:38.841509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1154/2307] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:02:38 2022] PUT /identity/v3/projects/7fc64ff05f7240b3a9cb0f191195b8e3/users/13d1363fdfcf4d889f1ab927310e0374/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:38.847137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1660-634f-4694-89d6-cc9af9b6dd97 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:38.847137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1660-634f-4694-89d6-cc9af9b6dd97 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:38.847407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c1660-634f-4694-89d6-cc9af9b6dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:38.867959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-477c1660-634f-4694-89d6-cc9af9b6dd97 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:38.869096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-477c1660-634f-4694-89d6-cc9af9b6dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:38.927897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1154/2308] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:38.937537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:38.939306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:38.962741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:38.999784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac9de0-8a7d-4b09-9bfb-9d2797e8053c admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:39.004027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:39.004634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:39.004920 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:39.005195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:39.014145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:39.015616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:39.025749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1155/2309] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:40.242576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0c3e-d15a-4b92-a3c0-df497db5cc34 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:40.243136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0c3e-d15a-4b92-a3c0-df497db5cc34 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:40.243245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a0c3e-d15a-4b92-a3c0-df497db5cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:40.264894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-366a0c3e-d15a-4b92-a3c0-df497db5cc34 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:40.266125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-366a0c3e-d15a-4b92-a3c0-df497db5cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:40.293385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:40.340634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1155/2310] 10.222.0.72 () {58 vars in 967 bytes} [Mon Sep 26 18:02:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:40.356784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:40.359774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:40.458971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2111e5-5577-4a63-97a3-8cd2b88ec737 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:40.464753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:40.465372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:40.465654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:40.466001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:40.497028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:40.503540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:40.515037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1156/2311] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:02:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:02:43.090425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.090915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.091077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.121227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] MFA Rules not processed for user `13d1363fdfcf4d889f1ab927310e0374`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:43.122391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:43.196800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1156/2312] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:43.201708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:43.215557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22aa00-4f5b-4649-bb88-a0e62c40635b service swift] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:43.220812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:43.222069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.222419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.222759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.223502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:43.226188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:43.264745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1157/2313] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:02:43 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:02:43.268894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:43.281189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efa67-7ceb-4c1c-857a-576032bc7d00 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:43.344570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:43.345199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.345479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.345752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.347345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:43.349308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:43.372358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1157/2314] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:02:43 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:02:43.376282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:43.389119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e833b5-658a-4884-9255-b1090b96f960 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:43.394907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:43.395758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.396125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.396479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.397227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:43.399337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:43.414883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1158/2315] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:43.416737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:43.428146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff77be-757a-433e-ae7e-4522b7ad04d5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:43.432797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:43.433455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.433739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.434016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] PATH_INFO: `/v3/projects/2b32003d56574fe082f58431b531a350/users/a63f0f59c26a4bd98743f0eaaf7bd94c/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.449650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b32003d56574fe082f58431b531a350, user_id=a63f0f59c26a4bd98743f0eaaf7bd94c, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:43.452356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:43.466993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1158/2316] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:02:43 2022] PUT /identity/v3/projects/2b32003d56574fe082f58431b531a350/users/a63f0f59c26a4bd98743f0eaaf7bd94c/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:43.471640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:43.482071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5ce7c-4da1-45d0-a88b-01650923049e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:43.560653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9fa50-5b38-4115-819b-8d5c782d3170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:43.562110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9fa50-5b38-4115-819b-8d5c782d3170 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.562110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9fa50-5b38-4115-819b-8d5c782d3170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.562494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9fa50-5b38-4115-819b-8d5c782d3170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.563374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9fa50-5b38-4115-819b-8d5c782d3170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:43.565643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9fa50-5b38-4115-819b-8d5c782d3170 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:43.577427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1159/2317] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:02:43.581644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:43.592379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeaab5-323b-4ab3-b322-f7735950a4d3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:43.598019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:43.598875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:43.599145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:43.599425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] PATH_INFO: `/v3/projects/2b32003d56574fe082f58431b531a350/users/a63f0f59c26a4bd98743f0eaaf7bd94c/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:43.601455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b32003d56574fe082f58431b531a350, user_id=a63f0f59c26a4bd98743f0eaaf7bd94c, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:43.604341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:43.617015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1159/2318] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:02:43 2022] PUT /identity/v3/projects/2b32003d56574fe082f58431b531a350/users/a63f0f59c26a4bd98743f0eaaf7bd94c/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:02:48.226669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae29a40-212c-4641-a0e5-462b0af8bd5c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:48.227196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae29a40-212c-4641-a0e5-462b0af8bd5c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:48.227196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae29a40-212c-4641-a0e5-462b0af8bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:48.270382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-aae29a40-212c-4641-a0e5-462b0af8bd5c None None] MFA Rules not processed for user `a63f0f59c26a4bd98743f0eaaf7bd94c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:48.271709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aae29a40-212c-4641-a0e5-462b0af8bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:48.276204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:48.350792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1160/2319] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:48.365522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:02:48.367352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:48.390179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:48.459300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbea553-e616-41c4-b340-32073e6ccd3d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:02:48.465079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48e041-6ea2-404b-b5b5-0d3d0c1e1bb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:02:48.466416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48e041-6ea2-404b-b5b5-0d3d0c1e1bb5 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:48.466639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48e041-6ea2-404b-b5b5-0d3d0c1e1bb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:48.466996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48e041-6ea2-404b-b5b5-0d3d0c1e1bb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:48.538379 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48e041-6ea2-404b-b5b5-0d3d0c1e1bb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:02:48.540010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48e041-6ea2-404b-b5b5-0d3d0c1e1bb5 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:02:48.550078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1160/2320] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:02:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:02:49.924992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00825fe6-1736-4f0c-80ef-c5730164ed92 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:02:49.925714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00825fe6-1736-4f0c-80ef-c5730164ed92 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:02:49.925714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00825fe6-1736-4f0c-80ef-c5730164ed92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:02:49.945359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-00825fe6-1736-4f0c-80ef-c5730164ed92 None None] MFA Rules not processed for user `13d1363fdfcf4d889f1ab927310e0374`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:02:49.946507 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-00825fe6-1736-4f0c-80ef-c5730164ed92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:02:49.953855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1161/2321] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:02:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:02:58.545187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=49 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:02:59.938816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=30 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:10.125096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.125699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.125699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.160737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] MFA Rules not processed for user `008e6c2c897741889510e6bf5181a9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:10.162744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:10.237329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1161/2322] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.555210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2b218-37dc-43f5-857f-8a78e2ab0bae None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.555748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2b218-37dc-43f5-857f-8a78e2ab0bae None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.555748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2b218-37dc-43f5-857f-8a78e2ab0bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.575380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8af2b218-37dc-43f5-857f-8a78e2ab0bae None None] MFA Rules not processed for user `276174ef02b1409b840a0de13aa4095d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.576652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8af2b218-37dc-43f5-857f-8a78e2ab0bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.632116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1162/2323] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:20.229326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=25 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:22.625024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=25 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:31.166912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:31.225899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67eaa8-2570-4539-8741-d20f226f81cd None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:31.229574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:31.230195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.230476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.230750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.241262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:31.242707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:31.255378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1162/2324] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:41.249099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=26 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:42.953910 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:42.954660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:42.954660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:42.977866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] MFA Rules not processed for user `4bbd5a38a6ce472790128a37f62d472b`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:42.979380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:43.049652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1163/2325] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:03:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:53.040941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=25 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:54.741730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:54.743215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:54.755659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d46c72-b332-49de-b9d1-21d598d812fa service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:54.759315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:54.759930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:54.760241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:54.760518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:54.762252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:54.777891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:54.779386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:54.791462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1163/2326] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:59.340853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:59.351879 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdc64a-9d83-4484-8e78-8c0e7818fa87 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:59.355558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:59.356305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.356587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.356860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:59.358646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:59.372744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:59.374203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.385722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1164/2327] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:01.770690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:01.782464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404523-5c20-4f3f-8f62-8cad5b21d4bc service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:01.789970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:01.790589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:01.790894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:01.791174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:01.792976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:01.806170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:01.807827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:01.818892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1164/2328] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:09.380711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:11.813372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=12 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:23.537207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.547096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a4f42-3b4c-4c3e-8cae-1641128e97be service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.552195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.552947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.553339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.553711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.555845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.642289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.644793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.656968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1165/2329] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:24.672605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.694050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0bc7-e5d2-475f-8a78-9d4864c22ff5 service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.704389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.705251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.705744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.706378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.709963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:24.727911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.729519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.742551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1165/2330] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:27.846695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:27.848833 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:27.861619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8556136-36a6-40ef-84c7-8c3dd192f946 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:27.877335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:27.879122 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:27.879877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:27.880362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] PATH_INFO: `/v3/users/008e6c2c897741889510e6bf5181a9c6` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:27.882023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008e6c2c897741889510e6bf5181a9c6)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:27.884339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:27.949416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008e6c2c897741889510e6bf5181a9c6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.950559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008e6c2c897741889510e6bf5181a9c6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.951533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008e6c2c897741889510e6bf5181a9c6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.963167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008e6c2c897741889510e6bf5181a9c6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.981403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-552e6344-91ef-4681-a48f-db97256ef2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008e6c2c897741889510e6bf5181a9c6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.996063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1166/2331] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:27 2022] DELETE /identity/v3/users/008e6c2c897741889510e6bf5181a9c6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:28.214077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.226683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b8f8e-0912-4caf-b169-578656936087 service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.303409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.304492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.304998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.305489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] PATH_INFO: `/v3/projects/93579356e4b54de5a06d395cd499ca13` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.307734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93579356e4b54de5a06d395cd499ca13)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.311552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.381988 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93579356e4b54de5a06d395cd499ca13', 'request_id': 'req-99609d76-e180-4cc7-b5d4-1567da891d78'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:28.391117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93579356e4b54de5a06d395cd499ca13', 'request_id': 'req-99609d76-e180-4cc7-b5d4-1567da891d78'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:28.402483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93579356e4b54de5a06d395cd499ca13', 'request_id': 'req-99609d76-e180-4cc7-b5d4-1567da891d78'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:28.419501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] The token cache is being invalidate because project 93579356e4b54de5a06d395cd499ca13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:28.419824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:28.420412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-99609d76-e180-4cc7-b5d4-1567da891d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:28.421959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1166/2332] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:28 2022] DELETE /identity/v3/projects/93579356e4b54de5a06d395cd499ca13 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:28.427221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.427646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.427996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.449217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:28.450959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:28.520775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1167/2333] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:28.529663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.530172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.530698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.557769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:28.559036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:28.567886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1167/2334] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:28.572542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.585905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad354c-e7b8-4777-9db8-97cfd8bc8c59 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.593164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.594185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.594686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.595342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.596458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:28.597626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.600682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.603597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:28.626095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1168/2335] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:28.632856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.642999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95d4a3-e71d-4549-8da9-1532c2c32ef7 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.648545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.649142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.649420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.649691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.650326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.652265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.695850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1168/2336] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:28 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:28.702054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.715765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c42ade-0524-487a-813d-7e3cf51b42ac admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.784024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.784905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.785186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.785459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.786098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.788081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.811814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1169/2337] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:28 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:28.818067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.831198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184dd91-4df4-48e4-bd51-64be2b80cdf9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.843637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.844726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.845242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.845716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.847437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.850536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.870283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1169/2338] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:28.874535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.888176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446f125-1c71-476f-97a7-f2c3799aace0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.893662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966192c3-5768-4d38-913c-32d8137477d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.894731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966192c3-5768-4d38-913c-32d8137477d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.895143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966192c3-5768-4d38-913c-32d8137477d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.895527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966192c3-5768-4d38-913c-32d8137477d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.896334 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966192c3-5768-4d38-913c-32d8137477d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.898108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966192c3-5768-4d38-913c-32d8137477d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.913368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1170/2339] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:28.917964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.930415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc344f66-48ab-494d-a0bb-3d9d703967d6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.936468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.937284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.937746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.938070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] PATH_INFO: `/v3/projects/e2365057277448c7b8a35f966f03c654/users/f6c6b3e9ce3048159ec39edd44bdf5f2/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.956088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2365057277448c7b8a35f966f03c654, user_id=f6c6b3e9ce3048159ec39edd44bdf5f2, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.959785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.990793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1170/2340] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:28 2022] PUT /identity/v3/projects/e2365057277448c7b8a35f966f03c654/users/f6c6b3e9ce3048159ec39edd44bdf5f2/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:29.004190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.004677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.005146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.057003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] MFA Rules not processed for user `f6c6b3e9ce3048159ec39edd44bdf5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:29.059089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:29.156535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1171/2341] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:29.164259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.188172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:29.252013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecb667-bfa6-4547-b600-f0ddfce7d081 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.259330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.260128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.260501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.260867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.261591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.263766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.294310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1171/2342] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:29 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:29.300823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.312547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce22c6-ff0f-465f-9b43-b72f69c3aa08 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.372501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.373112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.373387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.373743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.374385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.376355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.386805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:29.406429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1172/2343] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:29 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:29.413726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.432174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458930d-e027-44f7-bec8-f2e1318ae92a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.439812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.440745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.441245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.441725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.443091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.445188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.460091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1172/2344] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:29.466445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.483928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ae7b-23b4-4fb9-b577-50d50a94c7fa admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.492519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.493667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.494171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.494670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] PATH_INFO: `/v3/projects/6c4f981eb0934f9294775a619a317148/users/4196bc38b1744dc4baf4aeef89eb37b0/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.523120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4f981eb0934f9294775a619a317148, user_id=4196bc38b1744dc4baf4aeef89eb37b0, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.528341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.547527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1173/2345] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:29 2022] PUT /identity/v3/projects/6c4f981eb0934f9294775a619a317148/users/4196bc38b1744dc4baf4aeef89eb37b0/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:29.552811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.568963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ce7b2-5169-4e88-af2b-c89d2c83f79a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.642986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c897a-191e-4dcd-9dfe-fe01f050edcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.643605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c897a-191e-4dcd-9dfe-fe01f050edcd admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.644475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c897a-191e-4dcd-9dfe-fe01f050edcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.644752 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c897a-191e-4dcd-9dfe-fe01f050edcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.645407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c897a-191e-4dcd-9dfe-fe01f050edcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.647045 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c897a-191e-4dcd-9dfe-fe01f050edcd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.657848 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1173/2346] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:29.663512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.674591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb3078-df02-424b-b776-a9aa7a306495 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.680734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.681505 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.681829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.682143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] PATH_INFO: `/v3/projects/6c4f981eb0934f9294775a619a317148/users/4196bc38b1744dc4baf4aeef89eb37b0/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.684210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4f981eb0934f9294775a619a317148, user_id=4196bc38b1744dc4baf4aeef89eb37b0, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.687230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.699238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1174/2347] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:29 2022] PUT /identity/v3/projects/6c4f981eb0934f9294775a619a317148/users/4196bc38b1744dc4baf4aeef89eb37b0/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:29.710637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.711019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.711344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.742972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] MFA Rules not processed for user `4196bc38b1744dc4baf4aeef89eb37b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:29.744219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:29.821138 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1174/2348] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:29.837616 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.839558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:29.919739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aa329-795d-481f-8f1b-0e6bfc6327f0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.925995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80681846-48e8-4168-98dd-508f75e5cef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.926982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80681846-48e8-4168-98dd-508f75e5cef3 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.927435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80681846-48e8-4168-98dd-508f75e5cef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.927875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80681846-48e8-4168-98dd-508f75e5cef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.941654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80681846-48e8-4168-98dd-508f75e5cef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.943318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80681846-48e8-4168-98dd-508f75e5cef3 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.956061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1175/2349] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:29.999396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.017827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:30.123207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7a4ce-4353-4e89-91ad-0c6f8a922287 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.127805 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.128812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.129098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.129373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] PATH_INFO: `/v3/users/f6c6b3e9ce3048159ec39edd44bdf5f2` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.130568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c6b3e9ce3048159ec39edd44bdf5f2)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.132552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:30.204165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c6b3e9ce3048159ec39edd44bdf5f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:30.204993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c6b3e9ce3048159ec39edd44bdf5f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:30.216432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c6b3e9ce3048159ec39edd44bdf5f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:30.235112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c6b3e9ce3048159ec39edd44bdf5f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:30.236030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c6b3e9ce3048159ec39edd44bdf5f2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:30.262886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1175/2350] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:29 2022] DELETE /identity/v3/users/f6c6b3e9ce3048159ec39edd44bdf5f2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:30.266553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.267103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.267574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.301966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:30.304051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:30.401594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1176/2351] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:30.413966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.415459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:30.498033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd75d2-8dbf-4ffd-a439-b2ca9625f31f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.502198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.502894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.503675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.503941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.515195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.516791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:30.527966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1176/2352] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:30.911392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.913392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:30.932072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da2efb-25d6-4e06-8215-7ba7f13a701b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.941522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.942590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.942868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.943257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] PATH_INFO: `/v3/projects/e2365057277448c7b8a35f966f03c654` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.944648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2365057277448c7b8a35f966f03c654)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.947176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.028485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2365057277448c7b8a35f966f03c654', 'request_id': 'req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.031708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:31.041847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2365057277448c7b8a35f966f03c654', 'request_id': 'req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.055278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2365057277448c7b8a35f966f03c654', 'request_id': 'req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.070256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] The token cache is being invalidate because project e2365057277448c7b8a35f966f03c654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:31.070607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.071226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.073053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1177/2353] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:30 2022] DELETE /identity/v3/projects/e2365057277448c7b8a35f966f03c654 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:31.079587 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:31.098299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783542c-08a7-4cfc-95ad-417ae5d5be32 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:31.149781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:31.193001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:31.193612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.193946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.194377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] PATH_INFO: `/v3/users/4196bc38b1744dc4baf4aeef89eb37b0` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.195654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4196bc38b1744dc4baf4aeef89eb37b0)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:31.197866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.258100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4196bc38b1744dc4baf4aeef89eb37b0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.258935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4196bc38b1744dc4baf4aeef89eb37b0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.266662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4196bc38b1744dc4baf4aeef89eb37b0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.282388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4196bc38b1744dc4baf4aeef89eb37b0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.283226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-72e16de3-bc89-4b22-ad0e-d25cd54e7e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4196bc38b1744dc4baf4aeef89eb37b0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.298970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1177/2354] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:31 2022] DELETE /identity/v3/users/4196bc38b1744dc4baf4aeef89eb37b0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:31.654498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:31.656224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:31.680018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f802c0-b2f8-4ff1-a484-71a2dd3918fe admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:31.751000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:31.751000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.751000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.751618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] PATH_INFO: `/v3/projects/6c4f981eb0934f9294775a619a317148` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.752189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4f981eb0934f9294775a619a317148)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:31.754697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.810090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4f981eb0934f9294775a619a317148', 'request_id': 'req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.818215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4f981eb0934f9294775a619a317148', 'request_id': 'req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.827215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4f981eb0934f9294775a619a317148', 'request_id': 'req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.842476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] The token cache is being invalidate because project 6c4f981eb0934f9294775a619a317148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:31.842768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.843484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fc1a766f-2e36-4aa3-8e5f-a93e801b94e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:31.844998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1178/2355] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:31 2022] DELETE /identity/v3/projects/6c4f981eb0934f9294775a619a317148 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:31.850182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.850479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.850754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.872719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:31.873972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:31.950071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1178/2356] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:31.958582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.959448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.959448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.993216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:31.994504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:32.002806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1179/2357] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:32.007716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.018558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba68daf-2e98-486f-97c3-54742175e658 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.023793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.024418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.024699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.024969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.025644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:32.025942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.027804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.042598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1179/2358] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:32.046967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.057786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0b476-2284-4f8c-9de9-bbe935679513 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.063527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.064384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.064641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.065049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.066325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.069938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.116249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1180/2359] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:32 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:32.121346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.133450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd573f3c-3647-4bef-b5b5-d5bef711021f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.181575 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:32.222806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.223614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.223902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.224180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.224874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.226878 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.256461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1180/2360] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:32 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:32.257611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.276267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f3be2-cc67-4e77-85ff-4fd1a1ee1939 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.281887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.282579 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.282958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.283308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.284226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.286916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.298739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1181/2361] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:32.303522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.314377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103fd8c-c9d1-4e43-b83a-d72570114ada admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.319723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3401c-6aa4-4ab9-b968-2c386f6746b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.320339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3401c-6aa4-4ab9-b968-2c386f6746b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.320620 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3401c-6aa4-4ab9-b968-2c386f6746b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.320898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3401c-6aa4-4ab9-b968-2c386f6746b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.321551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3401c-6aa4-4ab9-b968-2c386f6746b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.323201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3401c-6aa4-4ab9-b968-2c386f6746b6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.335852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1181/2362] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:32.340171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.352732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f0c9b-aa0b-47cb-aa86-2ac20d7dbd2a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.358253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.359260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.359686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.360165 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] PATH_INFO: `/v3/projects/7aff09ed3dff4a91901957238250d9e1/users/9b428e1f0f8d40ffbc2fc48c2511dd1d/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.383202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aff09ed3dff4a91901957238250d9e1, user_id=9b428e1f0f8d40ffbc2fc48c2511dd1d, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.388737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.423072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1182/2363] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:32 2022] PUT /identity/v3/projects/7aff09ed3dff4a91901957238250d9e1/users/9b428e1f0f8d40ffbc2fc48c2511dd1d/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:32.443215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.443724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.444225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.495733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] MFA Rules not processed for user `9b428e1f0f8d40ffbc2fc48c2511dd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:32.497004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:32.600065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1182/2364] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:32.616173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:32.618939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:32.635040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:32.714962 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6044174-be5d-42b8-ab91-ed402d57ce3d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:32.720896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569083a-320e-4654-85f7-250de9d1914c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:32.721671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569083a-320e-4654-85f7-250de9d1914c service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:32.722765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569083a-320e-4654-85f7-250de9d1914c service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:32.723784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569083a-320e-4654-85f7-250de9d1914c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:32.734124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569083a-320e-4654-85f7-250de9d1914c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:32.736171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569083a-320e-4654-85f7-250de9d1914c service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:32.749367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1183/2365] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:33.456073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:33.459538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:33.716993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3507ab-e5ee-4c3e-8f6e-7f705f791ad9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:33.728867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:33.729960 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:33.730496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:33.731032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:33.734258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:33.762404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:33.764388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:33.797514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1183/2366] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:36.409166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:36.410580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:36.410807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:36.442057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] MFA Rules not processed for user `9979be70f604408d9b78050e3ba52f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:36.443702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:36.542897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1184/2367] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:37.023168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:37.024673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.077758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:37.106855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97307745-394c-48d0-9cff-ed5c2e2fcb26 service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:37.122576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:37.123654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:37.124177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:37.124863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:37.129397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.152373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:37.153928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:37.168675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1184/2368] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:37.823705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:37.828064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.943765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e3087-057d-484b-855b-c713d0002908 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:37.950324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:37.951232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:37.951512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:37.951782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:37.953633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.967699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:37.969380 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:37.981903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1185/2369] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:41.335616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.338660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:41.376721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9ba41-5407-45bc-a6a7-32ebe9cc6a64 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:41.384959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:41.386488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:41.387311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:41.388266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:41.396015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:41.421459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:41.425999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:41.447540 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1185/2370] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:42.647782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:42.651906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.688978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be696bb-1c77-4354-8ee4-357dc2786f63 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.732313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.733336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.733823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.734318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.737135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.760362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.762796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5f63e-4105-46ae-9507-3e975a69289c service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.785762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1186/2371] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:43.167176 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.212079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b8442-d55d-405d-a1aa-8d353a65aab0 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.222260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.223747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.224256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.224735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.228388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:43.338369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.342694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:43.356455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1186/2372] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:45.333819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:45.334640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:45.334640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:45.361000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] MFA Rules not processed for user `276174ef02b1409b840a0de13aa4095d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:45.364515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:45.413653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:45.466075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1187/2373] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:45.925654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:45.941781 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2ede9-d7e0-405f-a924-44a3077dbe51 service nova] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:45.947703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:45.948330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:45.948609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:45.948884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:45.951068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:45.962463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:45.963949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:45.976228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1187/2374] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:48.801071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.804049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:48.898900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca74f1-80aa-49a8-a08a-8b6088ce170a None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.906143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.907226 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.907713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.908187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.911976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:49.027607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:49.035073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b793285-29f8-47ad-9c8d-c1fff0759fc3 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:49.057715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1188/2375] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:50.852365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:50.954287 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408e91-2496-46e1-8f97-c2cc40b3b46f service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:50.960160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:50.962680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:50.962680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:50.962680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] PATH_INFO: `/v3/users/9b428e1f0f8d40ffbc2fc48c2511dd1d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:50.963722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b428e1f0f8d40ffbc2fc48c2511dd1d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:50.965697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.032511 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b428e1f0f8d40ffbc2fc48c2511dd1d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:51.033867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b428e1f0f8d40ffbc2fc48c2511dd1d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:51.037997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:51.054220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b428e1f0f8d40ffbc2fc48c2511dd1d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:51.090130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b428e1f0f8d40ffbc2fc48c2511dd1d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:51.091101 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b428e1f0f8d40ffbc2fc48c2511dd1d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:51.112342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1188/2376] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:50 2022] DELETE /identity/v3/users/9b428e1f0f8d40ffbc2fc48c2511dd1d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:51.117732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.118546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.118546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.142952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:51.145312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:51.260455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1189/2377] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:51.275608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.278041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:51.355312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed03407-29fc-4155-ac49-a4c579a64981 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.359562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3ccf6-72e7-4037-9f2c-c8b8fa1c9be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.360433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3ccf6-72e7-4037-9f2c-c8b8fa1c9be1 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.360707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3ccf6-72e7-4037-9f2c-c8b8fa1c9be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.361006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3ccf6-72e7-4037-9f2c-c8b8fa1c9be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.372185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3ccf6-72e7-4037-9f2c-c8b8fa1c9be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.373718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3ccf6-72e7-4037-9f2c-c8b8fa1c9be1 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.388271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1189/2378] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:51.848021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.850759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:51.863762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9355e0-b942-4115-bc00-96ece598e2f1 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.874213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.875082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.876391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.876391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] PATH_INFO: `/v3/projects/7aff09ed3dff4a91901957238250d9e1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.876878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aff09ed3dff4a91901957238250d9e1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.879715 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.952720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:51.992988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aff09ed3dff4a91901957238250d9e1', 'request_id': 'req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.004558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aff09ed3dff4a91901957238250d9e1', 'request_id': 'req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.027637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aff09ed3dff4a91901957238250d9e1', 'request_id': 'req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.050483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] The token cache is being invalidate because project 7aff09ed3dff4a91901957238250d9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:52.051170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.052640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-dcbe58c3-09d3-4015-b9ae-a44bfaf7fb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.054258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1190/2379] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:51 2022] DELETE /identity/v3/projects/7aff09ed3dff4a91901957238250d9e1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:52.060219 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.060990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.061583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.089357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:52.091033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.287696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1190/2380] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:52.307032 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.307582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.307945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.342325 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:52.344319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.360511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1191/2381] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:52.365736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.376828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef7ba8-7428-4079-ada5-a5d97c4c3fc2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.384264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.385224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.385685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.386149 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.387324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:52.387837 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.390967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.420650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1191/2382] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:52.421569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.437313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7350d-ae25-4110-9bbd-045ebe413792 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.448163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.450933 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.452384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.453234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.454648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.458167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.520940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1192/2383] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:52.522281 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.536761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77f50f-604f-42a1-9e0b-dde4d0f1f1c7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.656143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:52.679727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.686739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.688057 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.688551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.690127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.693295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.750948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.763180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1192/2384] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/users => generated 281 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:52.766416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ec802-621c-4d57-a463-cbbb1eb2f68b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.773247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.774187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.774670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.775211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.776042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.777949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.799388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.810723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1193/2385] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:52.823362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cbfe3-0b8e-4507-821e-a6cb4e6cff02 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.837052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac079-898d-4164-91b4-2df49c41a4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.838295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac079-898d-4164-91b4-2df49c41a4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.840426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac079-898d-4164-91b4-2df49c41a4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.840426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac079-898d-4164-91b4-2df49c41a4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.841280 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac079-898d-4164-91b4-2df49c41a4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.844167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac079-898d-4164-91b4-2df49c41a4f1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.864329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1193/2386] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:52.871830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.888538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452d9e0-7086-4b12-84a5-30cc44696acf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.897558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.898650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.899341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.899723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] PATH_INFO: `/v3/projects/7761d54433524d90a36a32de993614c7/users/1cde390702004888b1b5e50b30834cd8/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.920681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7761d54433524d90a36a32de993614c7, user_id=1cde390702004888b1b5e50b30834cd8, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.923508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.945152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1194/2387] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:52 2022] PUT /identity/v3/projects/7761d54433524d90a36a32de993614c7/users/1cde390702004888b1b5e50b30834cd8/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:52.961979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.962673 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.963198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.986968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.992494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.043020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] MFA Rules not processed for user `1cde390702004888b1b5e50b30834cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:53.045732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.161907 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5aa8a-55a1-4da5-ade2-18ba52187c01 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.177871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.178687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.179005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.179279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.181467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.210051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.213224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.230986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1194/2388] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:53.234180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1195/2389] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:53.297972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.302170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.395185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759bd3-d37d-410c-9ca9-b6194c94bb38 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.402294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988ef21-67ce-4bb8-8e68-f206bf328627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.402938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ef21-67ce-4bb8-8e68-f206bf328627 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.403371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ef21-67ce-4bb8-8e68-f206bf328627 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.404475 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ef21-67ce-4bb8-8e68-f206bf328627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.426938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988ef21-67ce-4bb8-8e68-f206bf328627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.431562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988ef21-67ce-4bb8-8e68-f206bf328627 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.445711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1195/2390] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:53.470805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.541988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:53.587869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783079e-2872-4236-8bf6-99e4493b250a service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.594201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.595501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.595975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.596487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] PATH_INFO: `/v3/users/1cde390702004888b1b5e50b30834cd8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.599203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cde390702004888b1b5e50b30834cd8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.604564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.709515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cde390702004888b1b5e50b30834cd8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.710460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cde390702004888b1b5e50b30834cd8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.711156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cde390702004888b1b5e50b30834cd8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.724392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cde390702004888b1b5e50b30834cd8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.746401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cde390702004888b1b5e50b30834cd8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.789814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1196/2391] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:53 2022] DELETE /identity/v3/users/1cde390702004888b1b5e50b30834cd8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:53.794339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.794660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.794999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.816364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:53.817559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.934951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1196/2392] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:53.947089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.949141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:54.075940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1571c-f5aa-4585-8153-5aaced4c32fa admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:54.083428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efb6fe-fbd0-45cd-a6aa-147f50dd4d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:54.084193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efb6fe-fbd0-45cd-a6aa-147f50dd4d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:54.084606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efb6fe-fbd0-45cd-a6aa-147f50dd4d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:54.085081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efb6fe-fbd0-45cd-a6aa-147f50dd4d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:54.103243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efb6fe-fbd0-45cd-a6aa-147f50dd4d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:54.105632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efb6fe-fbd0-45cd-a6aa-147f50dd4d6e service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:54.166792 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1197/2393] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:54.731262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:54.733679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:54.756133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8019-f50b-4d8e-af83-fdf3099f7248 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:54.770970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:54.771974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:54.772543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:54.772958 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] PATH_INFO: `/v3/projects/7761d54433524d90a36a32de993614c7` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:54.774702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7761d54433524d90a36a32de993614c7)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:54.777213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:54.779700 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:54.886915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7761d54433524d90a36a32de993614c7', 'request_id': 'req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.894898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7761d54433524d90a36a32de993614c7', 'request_id': 'req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.907229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7761d54433524d90a36a32de993614c7', 'request_id': 'req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.933808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] The token cache is being invalidate because project 7761d54433524d90a36a32de993614c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:54.934739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.937266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9cb5095c-4d15-4de2-91e7-4c616f0ea3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.947217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1197/2394] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:54 2022] DELETE /identity/v3/projects/7761d54433524d90a36a32de993614c7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:54.951457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:54.953119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:54.954034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.001150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:55.002341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:55.103057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:55.123438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1198/2395] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:55.132122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.132463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.132646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.168091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:55.170198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:55.191010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1198/2396] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:55.194228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.207007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3d759-05ef-4163-8646-f7d01f168347 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.213551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.214187 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.214469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.214750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.215530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:55.215840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.217760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.234025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1199/2397] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:55.239569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.251105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84361696-879e-442c-b4f5-65e85d01f81a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.258731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.260558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.260558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.260558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.261039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.263269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.313034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1199/2398] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:55 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:55.318470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.332837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639a976-32b9-4858-9838-a48d564d2ff7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.450470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.451518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.451886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.452223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.453904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.455216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.484826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1200/2399] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:55 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:55.490905 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.510007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9246aaa-49f6-4a38-b8e5-1ca18c363f4d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.519835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.520832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.521264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.523702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.523702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.525307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.537215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1200/2400] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:55.541687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.559932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cac71-9ac8-4420-b097-f4141d2a98f4 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.567086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083ef11-f6b0-417b-bd0e-663cb5db659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.567835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083ef11-f6b0-417b-bd0e-663cb5db659f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.568185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083ef11-f6b0-417b-bd0e-663cb5db659f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.569175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083ef11-f6b0-417b-bd0e-663cb5db659f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.569900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083ef11-f6b0-417b-bd0e-663cb5db659f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.572768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083ef11-f6b0-417b-bd0e-663cb5db659f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.595190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1201/2401] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:55.599481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.615230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0983dd-8431-4140-941e-61398e50424c admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.625496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.627080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.627748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.631178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] PATH_INFO: `/v3/projects/6bad7be3b68d455e88dd65ca78feb7ec/users/717967b871ae426e9ffbfe2cc6e74339/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.674816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bad7be3b68d455e88dd65ca78feb7ec, user_id=717967b871ae426e9ffbfe2cc6e74339, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.680594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.703998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1201/2402] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:55 2022] PUT /identity/v3/projects/6bad7be3b68d455e88dd65ca78feb7ec/users/717967b871ae426e9ffbfe2cc6e74339/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:55.723832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.724280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.724771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.805988 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] MFA Rules not processed for user `717967b871ae426e9ffbfe2cc6e74339`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:55.807894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:55.926568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1202/2403] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:55.940303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.943724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:56.061202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71561262-22d2-4263-95e7-510e5be57de4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:56.066021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118be9f-41fd-4190-86f0-d190b55687d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:56.066898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118be9f-41fd-4190-86f0-d190b55687d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:56.067598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118be9f-41fd-4190-86f0-d190b55687d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:56.067969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118be9f-41fd-4190-86f0-d190b55687d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:56.089083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118be9f-41fd-4190-86f0-d190b55687d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:56.092127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118be9f-41fd-4190-86f0-d190b55687d4 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:56.112099 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1202/2404] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:59.215913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.313819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:59.342393 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103f5d9-c1c1-4641-bd0e-418479cc4363 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.348263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.349382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.349382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.349740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] PATH_INFO: `/v3/users/717967b871ae426e9ffbfe2cc6e74339` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.350743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=717967b871ae426e9ffbfe2cc6e74339)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.353019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.460951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717967b871ae426e9ffbfe2cc6e74339'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:59.462573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717967b871ae426e9ffbfe2cc6e74339'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:59.464687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717967b871ae426e9ffbfe2cc6e74339'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:59.477795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717967b871ae426e9ffbfe2cc6e74339'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:59.526916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717967b871ae426e9ffbfe2cc6e74339'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:59.561512 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1203/2405] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:59 2022] DELETE /identity/v3/users/717967b871ae426e9ffbfe2cc6e74339 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:59.567709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6246e-38f2-4fc9-a22e-1130fb3b39ae None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.568718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6246e-38f2-4fc9-a22e-1130fb3b39ae None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.569483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6246e-38f2-4fc9-a22e-1130fb3b39ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.606647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-03b6246e-38f2-4fc9-a22e-1130fb3b39ae None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:59.609175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-03b6246e-38f2-4fc9-a22e-1130fb3b39ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:59.651132 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:59.772313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1203/2406] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:59.785803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.795359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:59.962567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aa298-6628-485e-9466-654a2e46f7d7 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.995893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.997702 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.997986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.998873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.013606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.015212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.028542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1204/2407] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:00.835789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.836919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.837414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.888218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:00.889771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.024505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1204/2408] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:01.107696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.110014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.284713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9799634-924f-4fa9-8561-9e993da99309 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.295290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f38df-e2e7-440b-ba74-6c5dd88abf00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.295994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f38df-e2e7-440b-ba74-6c5dd88abf00 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.296272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f38df-e2e7-440b-ba74-6c5dd88abf00 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.296551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f38df-e2e7-440b-ba74-6c5dd88abf00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.308914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f38df-e2e7-440b-ba74-6c5dd88abf00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.310735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f38df-e2e7-440b-ba74-6c5dd88abf00 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.326482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1205/2409] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:05:01.739212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.740726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.775928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de0cdb-a62c-4bf8-8261-e38b49b906de None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.784320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.785096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.785636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.786173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] PATH_INFO: `/v3/projects/6bad7be3b68d455e88dd65ca78feb7ec` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.787903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bad7be3b68d455e88dd65ca78feb7ec)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.790466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.968359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bad7be3b68d455e88dd65ca78feb7ec', 'request_id': 'req-6f93fb69-11e7-40f0-8197-54e7161bd959'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:01.984928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bad7be3b68d455e88dd65ca78feb7ec', 'request_id': 'req-6f93fb69-11e7-40f0-8197-54e7161bd959'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:01.994222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bad7be3b68d455e88dd65ca78feb7ec', 'request_id': 'req-6f93fb69-11e7-40f0-8197-54e7161bd959'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.022646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] The token cache is being invalidate because project 6bad7be3b68d455e88dd65ca78feb7ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:02.023021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.023707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f93fb69-11e7-40f0-8197-54e7161bd959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.032978 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1205/2410] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:01 2022] DELETE /identity/v3/projects/6bad7be3b68d455e88dd65ca78feb7ec => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:02.035815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.036163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.036406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.059985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:02.094113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:02.095411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.234890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1206/2411] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:02.248513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.249326 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.249678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.283330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:02.285396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.315866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.325928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1206/2412] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:02.336071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a23754-d37a-4944-8812-ef4f65460456 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.350174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.353265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.353732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.362102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.364968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:02.365307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.367202 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.391536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.399499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1207/2413] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:02.407339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc57fd-031e-4403-839e-4a80d3667d5a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.421243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.422103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.422392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.422803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.423684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.429188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.491302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1207/2414] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:05:02.495431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.515152 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c9d9a-5c67-4d0f-a226-9aeb1aa16e91 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.642119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.642888 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.644130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.644302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.644975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.647029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.674051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1208/2415] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:05:02.678894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.695680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13abfcf-ab0f-45cf-a7f9-d5c7332b5436 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.701181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.701809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.702238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.702566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.703866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.705510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.730448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1208/2416] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:02.734277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.765247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfb5eb-1a0f-4958-beb3-f54a481f766f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.781957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4397fcd-4aaa-4a94-b827-d7a19ead5cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.783204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4397fcd-4aaa-4a94-b827-d7a19ead5cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.783809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4397fcd-4aaa-4a94-b827-d7a19ead5cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.784384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4397fcd-4aaa-4a94-b827-d7a19ead5cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.785901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4397fcd-4aaa-4a94-b827-d7a19ead5cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.787804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4397fcd-4aaa-4a94-b827-d7a19ead5cd7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.808322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1209/2417] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:02.809430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.822753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0907-cb37-4521-b3b1-19d8bff68587 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.834366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.835815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.835815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.835815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] PATH_INFO: `/v3/projects/647426425d664632871eb7c2014db2bf/users/4f998f2e75e840928ec82fe919512f2c/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.864612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647426425d664632871eb7c2014db2bf, user_id=4f998f2e75e840928ec82fe919512f2c, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.868959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.874259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:02.913413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1209/2418] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:02 2022] PUT /identity/v3/projects/647426425d664632871eb7c2014db2bf/users/4f998f2e75e840928ec82fe919512f2c/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:02.916199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.916541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.916932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.991694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] MFA Rules not processed for user `4f998f2e75e840928ec82fe919512f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:02.993208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:03.154208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:03.163685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1210/2419] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:03.176881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:03.179413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:03.285147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640343f6-bd83-43a7-ab34-edb096d77f24 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:03.297600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38d889-a656-4311-b569-b534524fd198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:03.298377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d889-a656-4311-b569-b534524fd198 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:03.298823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d889-a656-4311-b569-b534524fd198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:03.299190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d889-a656-4311-b569-b534524fd198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:03.313079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38d889-a656-4311-b569-b534524fd198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:03.314581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38d889-a656-4311-b569-b534524fd198 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:03.328677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1210/2420] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:05:03.376806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:03.538052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19540e0-6010-46d4-846c-5cfa41e9ba53 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:03.549240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:03.549240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:03.549240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:03.549240 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] PATH_INFO: `/v3/users/4f998f2e75e840928ec82fe919512f2c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:03.551189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f998f2e75e840928ec82fe919512f2c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:03.578236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:03.764638 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f998f2e75e840928ec82fe919512f2c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:03.766472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f998f2e75e840928ec82fe919512f2c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:03.768667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f998f2e75e840928ec82fe919512f2c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:03.785402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f998f2e75e840928ec82fe919512f2c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:03.818449 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-b1b0b31a-2aa8-4f03-bdd2-8b8e62e0f83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f998f2e75e840928ec82fe919512f2c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:03.861294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1211/2421] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:03 2022] DELETE /identity/v3/users/4f998f2e75e840928ec82fe919512f2c => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:03.866162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:03.867912 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:03.868501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:03.904353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:03.904898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:03.906400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:03.931256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:03.932830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:03.950045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] MFA Rules not processed for user `28195a63990344edb6ea31f1f4463a82`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:03.953157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:04.142590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1211/2422] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:04.161292 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1212/2423] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:04.168016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:04.170417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:04.328265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c60bb1-cce3-4c7f-8cc8-83966b6b5901 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:04.336120 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3be85-3dba-4ef9-b810-19c2e9782e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:04.336904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3be85-3dba-4ef9-b810-19c2e9782e54 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:04.337285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3be85-3dba-4ef9-b810-19c2e9782e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:04.337678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3be85-3dba-4ef9-b810-19c2e9782e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:04.372452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3be85-3dba-4ef9-b810-19c2e9782e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:04.374164 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3be85-3dba-4ef9-b810-19c2e9782e54 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:04.388040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1212/2424] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:05.429953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:05.432568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:05.475628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431936d-7a1f-456f-b9b6-ffaade931efd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:05.491081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:05.492432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:05.492721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:05.493141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] PATH_INFO: `/v3/projects/647426425d664632871eb7c2014db2bf` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:05.499756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=647426425d664632871eb7c2014db2bf)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:05.504956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:05.594526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:05.679012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647426425d664632871eb7c2014db2bf', 'request_id': 'req-df555da3-ee63-4271-a5c2-da35479ec010'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.696358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647426425d664632871eb7c2014db2bf', 'request_id': 'req-df555da3-ee63-4271-a5c2-da35479ec010'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.713996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647426425d664632871eb7c2014db2bf', 'request_id': 'req-df555da3-ee63-4271-a5c2-da35479ec010'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.759310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] The token cache is being invalidate because project 647426425d664632871eb7c2014db2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:05.760828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.765834 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df555da3-ee63-4271-a5c2-da35479ec010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.771406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1213/2425] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:05 2022] DELETE /identity/v3/projects/647426425d664632871eb7c2014db2bf => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:05.791797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:05.792976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:05.793442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:05.819419 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:05.820971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:05.872824 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:05.958741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1213/2426] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:05.984472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:05.984761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:05.985041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.035158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:06.036840 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:06.056532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1214/2427] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:06.061314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.090866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5262c-10e4-4ba6-b639-8f52648f8e87 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.101440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.102377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.102674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.103098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.104151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:06.104498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.107098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.131893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1214/2428] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:06.136996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.157168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a9d56-a024-4a40-9747-2f877d7af7be None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.166235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.166904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.167196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.167463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.168225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.170898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.263836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1215/2429] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:06 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:05:06.300568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.327177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391bae5-252b-4a2b-a58b-268222a4cdbd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.469920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.472435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.473155 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.473619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.474592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.476996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.511295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1215/2430] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:06 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:05:06.516366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.536648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f52fda-ad01-4f58-a711-a4a83f14675f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.543192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.544200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.544554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.545164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.545506 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.548052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.591684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1216/2431] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:06.595961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.609769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30fd87-6a42-4f0e-b4d1-44da724601b1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.629484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20239939-4a8d-439a-aac4-d5dc8b36ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.630135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20239939-4a8d-439a-aac4-d5dc8b36ebf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.630413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20239939-4a8d-439a-aac4-d5dc8b36ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.630685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20239939-4a8d-439a-aac4-d5dc8b36ebf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.631420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20239939-4a8d-439a-aac4-d5dc8b36ebf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.633139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20239939-4a8d-439a-aac4-d5dc8b36ebf6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.646346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1216/2432] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:06.651022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.668831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32231646-8adc-49c7-ab6f-f0360bb4b75d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.674323 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.675654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.675941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.676239 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] PATH_INFO: `/v3/projects/2f455e494d9a42b8b29f7cbac3eaf17f/users/a581aab644c3460693a2d42061c18bc8/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.696017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f455e494d9a42b8b29f7cbac3eaf17f, user_id=a581aab644c3460693a2d42061c18bc8, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.700587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.745137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1217/2433] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:06 2022] PUT /identity/v3/projects/2f455e494d9a42b8b29f7cbac3eaf17f/users/a581aab644c3460693a2d42061c18bc8/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:06.763533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf86b8-1b0e-470c-bf42-8963b3e95587 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.763533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf86b8-1b0e-470c-bf42-8963b3e95587 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.763533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf86b8-1b0e-470c-bf42-8963b3e95587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.797473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-32bf86b8-1b0e-470c-bf42-8963b3e95587 None None] MFA Rules not processed for user `a581aab644c3460693a2d42061c18bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:06.799047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-32bf86b8-1b0e-470c-bf42-8963b3e95587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:06.947716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1217/2434] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:06.995252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.998502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:07.101232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80f4b8-69d0-4bd9-93ba-91a8ae8ff612 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.109466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d30a6-1b75-4c8d-bede-5f13c0be93a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.110408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d30a6-1b75-4c8d-bede-5f13c0be93a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.110951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d30a6-1b75-4c8d-bede-5f13c0be93a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.111389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d30a6-1b75-4c8d-bede-5f13c0be93a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.127743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d30a6-1b75-4c8d-bede-5f13c0be93a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.130514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d30a6-1b75-4c8d-bede-5f13c0be93a7 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.145095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1218/2435] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:08.909710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:08.910655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:08.911040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:08.939600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] MFA Rules not processed for user `a581aab644c3460693a2d42061c18bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:08.941211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:08.954926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1218/2436] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:17.138469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:18.931214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:29.354868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a6487-605d-46c1-959a-92e000267ef5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:29.354868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a6487-605d-46c1-959a-92e000267ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:29.355466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a6487-605d-46c1-959a-92e000267ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:29.399398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-412a6487-605d-46c1-959a-92e000267ef5 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:29.401003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-412a6487-605d-46c1-959a-92e000267ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:29.513434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1219/2437] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:05:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:29.529655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:29.533715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:29.656484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7b1ff-1080-4e0c-bb4f-187b923b4507 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:29.664904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:29.665708 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:29.666069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:29.666442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:29.679296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:29.681028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:29.701336 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1219/2438] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:05:39.497450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=25 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:39.696115 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=32 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:41.814647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f299ed-d4ca-4899-8e56-ae059a5b7e9f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:41.816485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f299ed-d4ca-4899-8e56-ae059a5b7e9f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:41.816485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f299ed-d4ca-4899-8e56-ae059a5b7e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:41.841787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d7f299ed-d4ca-4899-8e56-ae059a5b7e9f None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:41.843489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d7f299ed-d4ca-4899-8e56-ae059a5b7e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:41.856178 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1220/2439] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:05:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:41.875191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:41.889709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ec45-640e-46c7-8e6b-4733532741cd service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:41.896023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:41.896643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:41.896920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:41.897191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:41.916857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:41.918491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:41.928861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1220/2440] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 18:05:42.628762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce0a39-f967-4786-bdc8-de51135028f1 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:42.629022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce0a39-f967-4786-bdc8-de51135028f1 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:42.629425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce0a39-f967-4786-bdc8-de51135028f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:42.652017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-80ce0a39-f967-4786-bdc8-de51135028f1 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:42.653825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-80ce0a39-f967-4786-bdc8-de51135028f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:42.662307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1221/2441] 10.222.0.72 () {58 vars in 967 bytes} [Mon Sep 26 18:05:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:42.676601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:42.688152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4a7d2-1c43-41ab-96f6-014ac571d8ce service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:42.694466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:42.695126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:42.695402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:42.695668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:42.705801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:42.708933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:42.719768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1221/2442] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 18:05:43.054229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6c18-fb5d-4819-95e7-50243fdf97f9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.054771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6c18-fb5d-4819-95e7-50243fdf97f9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.054771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6c18-fb5d-4819-95e7-50243fdf97f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.080298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-73ba6c18-fb5d-4819-95e7-50243fdf97f9 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:43.083265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-73ba6c18-fb5d-4819-95e7-50243fdf97f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:43.095321 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1222/2443] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:43.110381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:43.124395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd8865-34a4-4c38-9114-beacd71c0082 service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:43.131210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:43.132580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.132580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.132580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.141378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:43.143008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:43.153231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1222/2444] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:05:43.633865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.634309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.634777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.664029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:43.665181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:43.674101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1223/2445] 10.222.0.72 () {58 vars in 967 bytes} [Mon Sep 26 18:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:43.690601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:43.704773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5b71e-a30c-446e-926f-dc73d43644e5 service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:43.712639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496f401-9199-487e-a529-97a144d096ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:43.713657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496f401-9199-487e-a529-97a144d096ab service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.714152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496f401-9199-487e-a529-97a144d096ab service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.714638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496f401-9199-487e-a529-97a144d096ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.725447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496f401-9199-487e-a529-97a144d096ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:43.728199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496f401-9199-487e-a529-97a144d096ab service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:43.742054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1223/2446] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 18:05:46.286601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:46.402964 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba7d8b-fe96-4a7b-8bb7-4ff349fa0940 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:46.412166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:46.413280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:46.413769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:46.414255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] PATH_INFO: `/v3/users/a581aab644c3460693a2d42061c18bc8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:46.416291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a581aab644c3460693a2d42061c18bc8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:46.420013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:46.515407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a581aab644c3460693a2d42061c18bc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:46.516474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a581aab644c3460693a2d42061c18bc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:46.517424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a581aab644c3460693a2d42061c18bc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:46.532412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a581aab644c3460693a2d42061c18bc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:46.549261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a581aab644c3460693a2d42061c18bc8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:46.567429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1224/2447] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:46 2022] DELETE /identity/v3/users/a581aab644c3460693a2d42061c18bc8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:46.573320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232be6fe-36ee-4cf2-8a6d-8460d2788211 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:46.573874 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232be6fe-36ee-4cf2-8a6d-8460d2788211 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:46.574681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232be6fe-36ee-4cf2-8a6d-8460d2788211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:46.599840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-232be6fe-36ee-4cf2-8a6d-8460d2788211 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:46.601092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-232be6fe-36ee-4cf2-8a6d-8460d2788211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:46.677449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1224/2448] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:46.689002 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:46.690730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:46.693559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:46.694041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:46.694569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:46.730933 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:46.732964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:46.791186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9086944-4bf5-4a5b-bd4b-079d869c420a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:46.795976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:46.796610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:46.796890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:46.797206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:46.810889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:46.817313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:46.818905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:46.833586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1225/2449] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:46.841761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1225/2450] 10.222.0.72 () {58 vars in 967 bytes} [Mon Sep 26 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:46.867568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:46.870467 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:47.040822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7a4b3-f419-4135-854a-be665b4ef21e service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:47.051635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84175bf1-a871-4ae5-882c-4254b90d4015 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:47.052665 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175bf1-a871-4ae5-882c-4254b90d4015 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:47.053151 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175bf1-a871-4ae5-882c-4254b90d4015 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:47.053634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175bf1-a871-4ae5-882c-4254b90d4015 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:47.070768 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84175bf1-a871-4ae5-882c-4254b90d4015 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:47.074081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84175bf1-a871-4ae5-882c-4254b90d4015 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:47.089859 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1226/2451] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:05:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:05:47.161846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:47.164358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:47.185888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa94357-c887-4700-b7c9-ca7333f1157e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:47.193501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:47.194424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:47.195226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:47.196397 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] PATH_INFO: `/v3/projects/2f455e494d9a42b8b29f7cbac3eaf17f` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:47.199651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f455e494d9a42b8b29f7cbac3eaf17f)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:47.202589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:47.297310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:47.301521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f455e494d9a42b8b29f7cbac3eaf17f', 'request_id': 'req-512277a3-1554-4dc1-9e0c-12bff618156c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.314344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f455e494d9a42b8b29f7cbac3eaf17f', 'request_id': 'req-512277a3-1554-4dc1-9e0c-12bff618156c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.323627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f455e494d9a42b8b29f7cbac3eaf17f', 'request_id': 'req-512277a3-1554-4dc1-9e0c-12bff618156c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.346052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] The token cache is being invalidate because project 2f455e494d9a42b8b29f7cbac3eaf17f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:47.346538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.348230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-512277a3-1554-4dc1-9e0c-12bff618156c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.349762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1226/2452] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:47 2022] DELETE /identity/v3/projects/2f455e494d9a42b8b29f7cbac3eaf17f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:57.084144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=34 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:57.341496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:05.604522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a31a-d0da-4377-91d5-ba7d7ddbf593 None None] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.605091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a31a-d0da-4377-91d5-ba7d7ddbf593 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.605542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87a31a-d0da-4377-91d5-ba7d7ddbf593 None None] PATH_INFO: `/` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.608684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1227/2453] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:06:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:06:05.619534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380d33a-5842-48b3-9ba4-02f5711104ba None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.620393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380d33a-5842-48b3-9ba4-02f5711104ba None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.620393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380d33a-5842-48b3-9ba4-02f5711104ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.649377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-d380d33a-5842-48b3-9ba4-02f5711104ba None None] MFA Rules not processed for user `b179c391ce574cd8836b43ea5bd94222`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:05.650487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d380d33a-5842-48b3-9ba4-02f5711104ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:05.730162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1227/2454] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:05.745720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a31a-d0da-4377-91d5-ba7d7ddbf593 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:05.747494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fe87a31a-d0da-4377-91d5-ba7d7ddbf593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:05.762603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87a31a-d0da-4377-91d5-ba7d7ddbf593 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:05.766883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674fca5-d2b0-418a-97f4-4785ca74b674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:05.767440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674fca5-d2b0-418a-97f4-4785ca74b674 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.767762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674fca5-d2b0-418a-97f4-4785ca74b674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.768079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674fca5-d2b0-418a-97f4-4785ca74b674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.778022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674fca5-d2b0-418a-97f4-4785ca74b674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:05.779839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674fca5-d2b0-418a-97f4-4785ca74b674 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:05.791084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1228/2455] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:05.928923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c6bf2-9e78-4432-ba9d-c23bf81b72f2 None None] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.929207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c6bf2-9e78-4432-ba9d-c23bf81b72f2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.929524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c6bf2-9e78-4432-ba9d-c23bf81b72f2 None None] PATH_INFO: `/` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.930962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1228/2456] 10.222.0.72 () {54 vars in 921 bytes} [Mon Sep 26 18:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:06:05.937336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39280-fc1e-4556-b503-018733410d3b None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.937724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39280-fc1e-4556-b503-018733410d3b None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.938086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39280-fc1e-4556-b503-018733410d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.968018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-aac39280-fc1e-4556-b503-018733410d3b None None] MFA Rules not processed for user `1adfb9b41a1d4d31bccf5cc87b955526`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:05.969278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aac39280-fc1e-4556-b503-018733410d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:06.076602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1229/2457] 10.222.0.72 () {58 vars in 1018 bytes} [Mon Sep 26 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:06.091325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c6bf2-9e78-4432-ba9d-c23bf81b72f2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:06.103970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c6bf2-9e78-4432-ba9d-c23bf81b72f2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:06.108083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:06.108981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:06.108981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:06.108981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:06.121268 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:06.122699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:06.133757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1229/2458] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:07.226042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39280-fc1e-4556-b503-018733410d3b None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:07.227650 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-aac39280-fc1e-4556-b503-018733410d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:07.243800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39280-fc1e-4556-b503-018733410d3b None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:07.249477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:07.250225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:07.250685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:07.251288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:07.254538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:07.369832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:07.371328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:07.385973 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1230/2459] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:12.144429 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.146241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:12.169381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac5034-6d68-442c-a3fc-dc4c7b889421 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.175965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.177371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.178021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.178569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] PATH_INFO: `/v3/users/276174ef02b1409b840a0de13aa4095d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.180890 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=276174ef02b1409b840a0de13aa4095d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.184023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.264910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276174ef02b1409b840a0de13aa4095d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.265570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276174ef02b1409b840a0de13aa4095d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.273780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276174ef02b1409b840a0de13aa4095d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.291647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276174ef02b1409b840a0de13aa4095d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.292828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2a9a99e2-fc97-43fe-947f-e6f1deb772ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276174ef02b1409b840a0de13aa4095d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.311945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1230/2460] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:12 2022] DELETE /identity/v3/users/276174ef02b1409b840a0de13aa4095d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:12.521465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.544843 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69733779-a36c-4aba-8f4f-e25bee687f02 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.616307 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.617417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.617913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.618199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] PATH_INFO: `/v3/projects/3f9772c3904046ee9778a870f4adc0ae` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.619501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9772c3904046ee9778a870f4adc0ae)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.621433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.645564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:12.688023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9772c3904046ee9778a870f4adc0ae', 'request_id': 'req-d3bdb74e-19c3-49ec-99f0-ba0471962158'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.696395 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9772c3904046ee9778a870f4adc0ae', 'request_id': 'req-d3bdb74e-19c3-49ec-99f0-ba0471962158'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.707302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9772c3904046ee9778a870f4adc0ae', 'request_id': 'req-d3bdb74e-19c3-49ec-99f0-ba0471962158'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.720822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] The token cache is being invalidate because project 3f9772c3904046ee9778a870f4adc0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:12.721188 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.722147 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d3bdb74e-19c3-49ec-99f0-ba0471962158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.724164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1231/2461] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:12 2022] DELETE /identity/v3/projects/3f9772c3904046ee9778a870f4adc0ae => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:12.730222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.730706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.731242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.760684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:12.762579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:12.831624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1231/2462] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:12.839724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.840007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.840344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.862791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:12.864746 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:12.874291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1232/2463] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:12.881710 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.897677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1674ae3-6a1a-4662-84c7-b50a4c2f3f34 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.904613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.905376 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.905687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.905994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.906649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:12.906987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.908607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.912456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:12.930767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1232/2464] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:12.932216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.946026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f636a-1fb2-43a7-a420-a47085f77460 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.953363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.954297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.954740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.955183 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.956493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.959906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.998703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1233/2465] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:12 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:13.004328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.021413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185acb3-e5d5-43b5-9a1b-6d7fe9321589 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.088560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.089085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.089369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.089640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.090272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.092233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.114634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1233/2466] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:13.118966 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.131728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b93f-3eba-4ed4-8ab7-3bc4ebfda3dc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.137045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.137658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.137954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.138219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.138927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.140543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.152278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1234/2467] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:13.156622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.168408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5357c5c-4261-48a0-9c67-b8f71348f94b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.173331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd73c1-4161-4386-878c-010fcaded767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.173934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd73c1-4161-4386-878c-010fcaded767 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.174337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd73c1-4161-4386-878c-010fcaded767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.175083 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd73c1-4161-4386-878c-010fcaded767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.175740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd73c1-4161-4386-878c-010fcaded767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.177337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd73c1-4161-4386-878c-010fcaded767 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.188965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1234/2468] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:13.194300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.205326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba529b-71a6-4c48-a50e-c30aa7c6c036 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.211499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.212399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.212830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.213207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] PATH_INFO: `/v3/projects/16c518713f85491dbac8b1757fc33d1a/users/ccefb124600f4ca3ab89ac9ef47a3661/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.230825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c518713f85491dbac8b1757fc33d1a, user_id=ccefb124600f4ca3ab89ac9ef47a3661, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.233693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.258287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1235/2469] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:13 2022] PUT /identity/v3/projects/16c518713f85491dbac8b1757fc33d1a/users/ccefb124600f4ca3ab89ac9ef47a3661/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:13.273498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.273775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.274043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.318116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] MFA Rules not processed for user `ccefb124600f4ca3ab89ac9ef47a3661`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:13.320257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.446913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1235/2470] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:13.452173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.571416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca38531-35d8-4e3c-a41c-1439fb67d793 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.578095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.578896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.579191 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.579466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.580372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.583058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.624851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1236/2471] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:13.632127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.646665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbda5c-d007-4529-8083-b5e123dc09a2 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.731063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.731865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.732314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.732668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.733515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.735685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.749613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:13.766323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1236/2472] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:13.772100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.790074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a4330-9e95-4186-a4d1-306d362d95b3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.798444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.799108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.800013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.800408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.801066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.803452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.818855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1237/2473] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:13.823968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.836108 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87446f-b73f-4d0a-ad88-c5a4a091c3cf admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.841206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.841872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.842154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.842428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] PATH_INFO: `/v3/projects/6fdd966032e146d29e18b36a3b5e0ba7/users/c0ec6347e0564343b6235b849bfe49ab/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.861711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdd966032e146d29e18b36a3b5e0ba7, user_id=c0ec6347e0564343b6235b849bfe49ab, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.866730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.888651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1237/2474] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:13 2022] PUT /identity/v3/projects/6fdd966032e146d29e18b36a3b5e0ba7/users/c0ec6347e0564343b6235b849bfe49ab/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:13.894708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.907267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75de9-34a0-4c07-afb7-976062ca0b1a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.919443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:13.989439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ff82f-0b69-4731-b98a-bff50dee56b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.990074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ff82f-0b69-4731-b98a-bff50dee56b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.990361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ff82f-0b69-4731-b98a-bff50dee56b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.990640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ff82f-0b69-4731-b98a-bff50dee56b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.991329 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ff82f-0b69-4731-b98a-bff50dee56b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.993044 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ff82f-0b69-4731-b98a-bff50dee56b7 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.007080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1238/2475] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:14.011946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.024009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7e69b-159b-45c7-913f-8558a3bbd5ec admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.028921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.029594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.029879 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.030190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] PATH_INFO: `/v3/projects/6fdd966032e146d29e18b36a3b5e0ba7/users/c0ec6347e0564343b6235b849bfe49ab/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.032298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fdd966032e146d29e18b36a3b5e0ba7, user_id=c0ec6347e0564343b6235b849bfe49ab, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.035541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.046714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1238/2476] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:14 2022] PUT /identity/v3/projects/6fdd966032e146d29e18b36a3b5e0ba7/users/c0ec6347e0564343b6235b849bfe49ab/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:14.059279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.059723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.060101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.094697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] MFA Rules not processed for user `c0ec6347e0564343b6235b849bfe49ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:14.095961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:14.179471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1239/2477] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:14.192967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.194784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:14.267981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0928a1-31d6-4164-96a9-871c75dfc6a8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.272461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.273085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.273371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.273648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.284106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.285558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.296315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1239/2478] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:14.398477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.412491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac5338-b5d4-47dd-b098-f254cdfc6fea None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.417272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef926f90-ddfc-476d-bcda-c106a46e3dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.417994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926f90-ddfc-476d-bcda-c106a46e3dcf service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.418397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926f90-ddfc-476d-bcda-c106a46e3dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.418788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926f90-ddfc-476d-bcda-c106a46e3dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.485805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef926f90-ddfc-476d-bcda-c106a46e3dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.488045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef926f90-ddfc-476d-bcda-c106a46e3dcf service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.498006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1240/2479] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:17.495250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:17.601713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc7df2-c990-4f55-834d-88b2e3e4bfe6 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:17.609374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:17.610655 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:17.610862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:17.611493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] PATH_INFO: `/v3/users/ccefb124600f4ca3ab89ac9ef47a3661` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:17.613585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccefb124600f4ca3ab89ac9ef47a3661)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:17.616182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:17.722317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccefb124600f4ca3ab89ac9ef47a3661'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.723638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccefb124600f4ca3ab89ac9ef47a3661'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.732789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccefb124600f4ca3ab89ac9ef47a3661'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.743198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:17.753213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccefb124600f4ca3ab89ac9ef47a3661'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.755186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccefb124600f4ca3ab89ac9ef47a3661'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.776249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1240/2480] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:17 2022] DELETE /identity/v3/users/ccefb124600f4ca3ab89ac9ef47a3661 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:17.781201 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:17.782056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:17.782056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:17.819399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:17.821588 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:17.867784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:17.933208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1241/2481] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:17.945292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:17.946945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:18.037887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d34ad4-fb5e-427f-aa23-9d8b40a3132e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:18.042908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:18.043689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:18.044188 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:18.044716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:18.056075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:18.057532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:18.068322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1241/2482] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:18.506623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:18.512782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:18.531073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854cc02-6f2d-4306-ad5a-d3470b03b834 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:18.537063 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:18.538623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:18.539098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:18.539298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] PATH_INFO: `/v3/projects/16c518713f85491dbac8b1757fc33d1a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:18.540531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c518713f85491dbac8b1757fc33d1a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:18.543690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:18.611235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c518713f85491dbac8b1757fc33d1a', 'request_id': 'req-69605d9f-7c0a-414d-9516-5ff56032cbe2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.618068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c518713f85491dbac8b1757fc33d1a', 'request_id': 'req-69605d9f-7c0a-414d-9516-5ff56032cbe2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.627186 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c518713f85491dbac8b1757fc33d1a', 'request_id': 'req-69605d9f-7c0a-414d-9516-5ff56032cbe2'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.639944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] The token cache is being invalidate because project 16c518713f85491dbac8b1757fc33d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:18.640309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.640994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.642571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1242/2483] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:18 2022] DELETE /identity/v3/projects/16c518713f85491dbac8b1757fc33d1a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:18.648215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:18.663445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee290cb-690d-463d-a7a3-dee52ade1ab4 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:18.734090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:18.735242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:18.735567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:18.736015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] PATH_INFO: `/v3/users/c0ec6347e0564343b6235b849bfe49ab` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:18.737823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ec6347e0564343b6235b849bfe49ab)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:18.740167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:18.798505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ec6347e0564343b6235b849bfe49ab'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.799894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ec6347e0564343b6235b849bfe49ab'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.808791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ec6347e0564343b6235b849bfe49ab'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.823904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ec6347e0564343b6235b849bfe49ab'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.824785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6c993b26-015d-49bc-97a3-34a88a209a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ec6347e0564343b6235b849bfe49ab'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:18.838103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:18.844025 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1242/2484] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:18 2022] DELETE /identity/v3/users/c0ec6347e0564343b6235b849bfe49ab => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:19.272308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.276949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:19.308726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605d9f-7c0a-414d-9516-5ff56032cbe2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.402657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.403808 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.404308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.404868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] PATH_INFO: `/v3/projects/6fdd966032e146d29e18b36a3b5e0ba7` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.407596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fdd966032e146d29e18b36a3b5e0ba7)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:19.411908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:19.474419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fdd966032e146d29e18b36a3b5e0ba7', 'request_id': 'req-d9784d9a-10cf-4916-bce3-98fe8171875c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.483775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fdd966032e146d29e18b36a3b5e0ba7', 'request_id': 'req-d9784d9a-10cf-4916-bce3-98fe8171875c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.489131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:19.495883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fdd966032e146d29e18b36a3b5e0ba7', 'request_id': 'req-d9784d9a-10cf-4916-bce3-98fe8171875c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.515347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] The token cache is being invalidate because project 6fdd966032e146d29e18b36a3b5e0ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:19.515827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.516822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d9784d9a-10cf-4916-bce3-98fe8171875c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.526938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1243/2485] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:19 2022] DELETE /identity/v3/projects/6fdd966032e146d29e18b36a3b5e0ba7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:19.527663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.527828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.528293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.567109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:19.568579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:19.684158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1243/2486] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:19.692102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.692391 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.692672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.726319 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:19.727473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:19.742039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1244/2487] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:19.747080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.764522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd4ad-d0b7-4ada-a8db-d6718acfae08 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.769537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.770530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.770866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.771417 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.771785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:19.772095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:19.773718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:19.791401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1244/2488] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:19.793621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.804581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b833ba9-c3b4-489d-a039-cccb5a6dd3eb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.809704 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.810344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.810800 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.811300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.812378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:19.814403 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:19.846317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1245/2489] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:19.852362 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.870777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70f05c-b0be-4768-9b04-f5c100f8cb1b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.933855 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.934797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.935174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.935456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.936110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:19.939178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:19.964325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1245/2490] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:19.970930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.991042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d07a15-1c2d-41f9-a598-959b7622dc2a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.998677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.999703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.000179 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.000652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.001760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.004948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.017445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1246/2491] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:20.021721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.032709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8161f-6eb0-4bd3-b5d3-a2059f4e7e02 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.038689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f049bc-d5a2-420f-bc14-85862a8c6157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.039344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f049bc-d5a2-420f-bc14-85862a8c6157 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.039629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f049bc-d5a2-420f-bc14-85862a8c6157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.039906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f049bc-d5a2-420f-bc14-85862a8c6157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.040554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f049bc-d5a2-420f-bc14-85862a8c6157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.042185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f049bc-d5a2-420f-bc14-85862a8c6157 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.057959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1246/2492] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:20.063803 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.074283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55079fb8-5b17-4703-9b31-0dfa4c106d28 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.082018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.083233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.083699 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.084322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] PATH_INFO: `/v3/projects/0cd273eaa11c4322909f0bbe54339b7b/users/b059757a3a3e4f729b3d1a6313d16ef2/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.104868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd273eaa11c4322909f0bbe54339b7b, user_id=b059757a3a3e4f729b3d1a6313d16ef2, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.107648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.123436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1247/2493] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:20 2022] PUT /identity/v3/projects/0cd273eaa11c4322909f0bbe54339b7b/users/b059757a3a3e4f729b3d1a6313d16ef2/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:20.135586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.135889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.136203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.168491 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] MFA Rules not processed for user `b059757a3a3e4f729b3d1a6313d16ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:20.170014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:20.233178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:20.249051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1247/2494] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:20.250330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.333284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6b9fd-8a2b-4c94-9b28-f193307fdc8b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.339780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.340399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.340685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.341192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.342365 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.345946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.380235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1248/2495] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:20 2022] POST /identity/v3/projects => generated 393 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:20.384367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.397683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55369f-fef7-4595-a8ef-09b01ae4c634 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.461861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.462478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.462762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.463074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.463724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.465647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.490776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1248/2496] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:20 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:20.495708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.513829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77194faf-fea3-47c0-a216-7bfde1daac59 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.521401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.522385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.522893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.523366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.524474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.527431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.540827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1249/2497] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:20.545799 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.556593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39778fe4-b986-4789-977e-fea6fb7fdc9f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.561658 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.562319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.562600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.562917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] PATH_INFO: `/v3/projects/1ae285ff9c55451193612119d1321117/users/4d8b3d78084b456f987107b9b188057d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.580844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae285ff9c55451193612119d1321117, user_id=4d8b3d78084b456f987107b9b188057d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.583828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.599075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1249/2498] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:20 2022] PUT /identity/v3/projects/1ae285ff9c55451193612119d1321117/users/4d8b3d78084b456f987107b9b188057d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:20.603162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.614412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe9e43-ad1b-44f3-9d5a-f9b43b96e672 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.669583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:20.679096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd62b2-ac72-4817-afdf-42c5e87ed0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.680031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd62b2-ac72-4817-afdf-42c5e87ed0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.680709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd62b2-ac72-4817-afdf-42c5e87ed0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.681457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd62b2-ac72-4817-afdf-42c5e87ed0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.682713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd62b2-ac72-4817-afdf-42c5e87ed0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.684934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd62b2-ac72-4817-afdf-42c5e87ed0d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.696470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1250/2499] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:20.701372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.712594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b119fef-1be4-46c7-8e19-ae2f22fe8ae7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.717931 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.718584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.718898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.719175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] PATH_INFO: `/v3/projects/1ae285ff9c55451193612119d1321117/users/4d8b3d78084b456f987107b9b188057d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.721938 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae285ff9c55451193612119d1321117, user_id=4d8b3d78084b456f987107b9b188057d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.724604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.736554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1250/2500] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:20 2022] PUT /identity/v3/projects/1ae285ff9c55451193612119d1321117/users/4d8b3d78084b456f987107b9b188057d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:20.753693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.753990 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.754268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.785935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] MFA Rules not processed for user `4d8b3d78084b456f987107b9b188057d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:20.787818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:20.895812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1251/2501] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:20.912500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.914386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:21.001515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65c56b-5205-4b7e-a863-cf6517f23539 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:21.015961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:21.016955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:21.017387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:21.017835 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:21.032736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:21.035285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:21.048807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1251/2502] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:21.110509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:21.112538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:21.133054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e1f0d-a9a3-4f56-a527-6f196963aa78 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:21.136739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4cc6d-6bb4-4a42-baee-a2b8c4a4b17e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:21.137443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4cc6d-6bb4-4a42-baee-a2b8c4a4b17e service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:21.137731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4cc6d-6bb4-4a42-baee-a2b8c4a4b17e service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:21.137995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4cc6d-6bb4-4a42-baee-a2b8c4a4b17e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:21.209123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4cc6d-6bb4-4a42-baee-a2b8c4a4b17e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:21.210871 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4cc6d-6bb4-4a42-baee-a2b8c4a4b17e service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:21.223042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1252/2503] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:24.895986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:24.951449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:24.965949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f163f-0ef1-478c-8f79-0e9eb30377f5 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:24.972260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:24.972969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:24.973286 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:24.973637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] PATH_INFO: `/v3/users/b059757a3a3e4f729b3d1a6313d16ef2` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:24.974998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b059757a3a3e4f729b3d1a6313d16ef2)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:24.977079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.036900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b059757a3a3e4f729b3d1a6313d16ef2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.037598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b059757a3a3e4f729b3d1a6313d16ef2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.045282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b059757a3a3e4f729b3d1a6313d16ef2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.060921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b059757a3a3e4f729b3d1a6313d16ef2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.062040 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b059757a3a3e4f729b3d1a6313d16ef2'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.083319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1252/2504] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:24 2022] DELETE /identity/v3/users/b059757a3a3e4f729b3d1a6313d16ef2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:25.088818 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.089358 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.089466 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.112435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:25.113860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:25.191790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1253/2505] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:25.203190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.204819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:25.278495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee12232-1b14-4c75-bcc5-4aa9b4480725 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.283249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.284005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.284813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.285091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.295662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.297111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.309185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1253/2506] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:25.659753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.661344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:25.675987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f67ea8-3854-4e1f-a735-5060ec5f21e9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.685415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.686328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.686610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.687168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] PATH_INFO: `/v3/projects/0cd273eaa11c4322909f0bbe54339b7b` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.689684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd273eaa11c4322909f0bbe54339b7b)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.693722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.713342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:25.778610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd273eaa11c4322909f0bbe54339b7b', 'request_id': 'req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.786900 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd273eaa11c4322909f0bbe54339b7b', 'request_id': 'req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.796806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd273eaa11c4322909f0bbe54339b7b', 'request_id': 'req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.814476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] The token cache is being invalidate because project 0cd273eaa11c4322909f0bbe54339b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:25.815053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.816442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.818830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1254/2507] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:25 2022] DELETE /identity/v3/projects/0cd273eaa11c4322909f0bbe54339b7b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:25.825093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.838747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9764e1-5304-4067-851e-da9f58cf7452 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.917058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.918160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.919296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.919296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] PATH_INFO: `/v3/users/4d8b3d78084b456f987107b9b188057d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.921259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8b3d78084b456f987107b9b188057d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.923650 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.942406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:25.988533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8b3d78084b456f987107b9b188057d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.989271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8b3d78084b456f987107b9b188057d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:25.997456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8b3d78084b456f987107b9b188057d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.010748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8b3d78084b456f987107b9b188057d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.011440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-465c73b5-1e62-44c3-92ab-c39f34e0f58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8b3d78084b456f987107b9b188057d'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.026043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1254/2508] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:25 2022] DELETE /identity/v3/users/4d8b3d78084b456f987107b9b188057d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:26.398428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:26.400718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:26.426511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d405cd-6a34-4e8e-a8c8-6e6051e9b3cd admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:26.506252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:26.507454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.507941 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.508636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] PATH_INFO: `/v3/projects/1ae285ff9c55451193612119d1321117` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.510617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae285ff9c55451193612119d1321117)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:26.513862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:26.581913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae285ff9c55451193612119d1321117', 'request_id': 'req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.591437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae285ff9c55451193612119d1321117', 'request_id': 'req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.602361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae285ff9c55451193612119d1321117', 'request_id': 'req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.619378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] The token cache is being invalidate because project 1ae285ff9c55451193612119d1321117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:26.620431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.621431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2e48cd77-e3b9-4faa-8e0d-192fa593e0b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:26.622953 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1255/2509] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:26 2022] DELETE /identity/v3/projects/1ae285ff9c55451193612119d1321117 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:26.629434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.629898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.630459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.661228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:26.663081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:26.739191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1255/2510] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:26.747609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.747926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.748203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.769357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:26.770591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:26.778685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1256/2511] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:26.783304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:26.794170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699076ab-4e0c-4301-a2b5-a9ea10af7c74 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:26.798712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:26.799385 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.800202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.800473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.801218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:26.801521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:26.803165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:26.822492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1256/2512] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:26.823600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:26.834770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98d6aa-4185-41ed-bc5f-97fe40302677 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:26.839642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:26.840267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.840778 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.840986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.841634 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:26.843596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:26.878474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1257/2513] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:26.882892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:26.896619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d53ea-ef20-44ad-b4f1-2159ad50ae07 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:26.974388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:26.975487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.976001 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.976635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.979039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:26.981103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.013656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1257/2514] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:27.019020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.025651 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:27.034256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01946ab-5640-4fb1-a161-421fb8fbd30b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.041524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.042134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.042417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.042684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.043570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.045231 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.057328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1258/2515] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:27.061808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.075147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd4ba7-11c0-49cc-85b5-8526a26c45b3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.083194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c03a88-231d-4ca4-bf81-47f498c7d03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.083819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c03a88-231d-4ca4-bf81-47f498c7d03a admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.084103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c03a88-231d-4ca4-bf81-47f498c7d03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.084373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c03a88-231d-4ca4-bf81-47f498c7d03a admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.085049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c03a88-231d-4ca4-bf81-47f498c7d03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.086651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c03a88-231d-4ca4-bf81-47f498c7d03a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.092997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:27.109347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1258/2516] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:27.115074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.133675 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9aac4-51fa-4c41-b43d-9498b6af3284 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.141504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.142620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.143658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.144119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] PATH_INFO: `/v3/projects/4203814ad37f43f1b26f89885f5a02ba/users/64b80bd1edb74f6db0dcbdfe966b10db/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.164534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4203814ad37f43f1b26f89885f5a02ba, user_id=64b80bd1edb74f6db0dcbdfe966b10db, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.167254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.182371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1259/2517] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:27 2022] PUT /identity/v3/projects/4203814ad37f43f1b26f89885f5a02ba/users/64b80bd1edb74f6db0dcbdfe966b10db/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:27.195383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.195856 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.196295 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.239709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] MFA Rules not processed for user `64b80bd1edb74f6db0dcbdfe966b10db`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:27.241172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:27.339030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1259/2518] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:27.345945 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.432721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22de1d9-7f65-48ab-a3fb-3043f2190378 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.440956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.441810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.442255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.442607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.443414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.445509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.478866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1260/2519] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:27 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:27.482166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.496031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd54de-e617-441a-bacd-2836a33edd31 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.576359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.576929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.577456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.577803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.578662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.581622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.608338 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1260/2520] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:27 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:27.612974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.626628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50080c09-8eb3-4ac2-8fff-992a93b5f9b8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.632118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.632743 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.633024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.633300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.633939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.635741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.648139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1261/2521] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:27.652839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.666308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e55a4-d6dd-43ff-a896-686ef29c77c4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.671866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.672538 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.672821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.673105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] PATH_INFO: `/v3/projects/93a1c03176734cf2ac61fef9a0cb39f8/users/1b655a134b954185aa268275adc9f2cb/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.700984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a1c03176734cf2ac61fef9a0cb39f8, user_id=1b655a134b954185aa268275adc9f2cb, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.704016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.719919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1261/2522] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:27 2022] PUT /identity/v3/projects/93a1c03176734cf2ac61fef9a0cb39f8/users/1b655a134b954185aa268275adc9f2cb/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:27.724633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.735814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b35b1d-8fe9-40b7-b3b9-36f56322726a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.801052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed33fd-3af7-4979-8335-8adb296d39e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.801673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed33fd-3af7-4979-8335-8adb296d39e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.801952 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed33fd-3af7-4979-8335-8adb296d39e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.802332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed33fd-3af7-4979-8335-8adb296d39e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.803037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed33fd-3af7-4979-8335-8adb296d39e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.804627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed33fd-3af7-4979-8335-8adb296d39e1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.816791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1262/2523] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:27.820987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.837229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd558c-0be3-418a-a49d-518cb6124c8f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.844443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.845483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.845940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.846400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] PATH_INFO: `/v3/projects/93a1c03176734cf2ac61fef9a0cb39f8/users/1b655a134b954185aa268275adc9f2cb/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.850907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a1c03176734cf2ac61fef9a0cb39f8, user_id=1b655a134b954185aa268275adc9f2cb, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.854166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.867831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1262/2524] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:27 2022] PUT /identity/v3/projects/93a1c03176734cf2ac61fef9a0cb39f8/users/1b655a134b954185aa268275adc9f2cb/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:27.881463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.881992 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.882465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.919098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] MFA Rules not processed for user `1b655a134b954185aa268275adc9f2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:27.919098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:27.952652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:28.009757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1263/2525] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:28.024319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:28.029702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:28.088735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:28.107022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f21bd-ea78-48da-a77a-5db1e358b37a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:28.110676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:28.111323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:28.111600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:28.112525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:28.125680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:28.128412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:28.138919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1263/2526] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:28.283382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:28.302904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85c73e-79fd-49aa-bbd0-3d3e69efee03 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:28.309560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e6a77-33db-40b8-b085-eee51f135c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:28.310417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6a77-33db-40b8-b085-eee51f135c7b service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:28.310926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6a77-33db-40b8-b085-eee51f135c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:28.311399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6a77-33db-40b8-b085-eee51f135c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:28.399219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e6a77-33db-40b8-b085-eee51f135c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:28.401212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e6a77-33db-40b8-b085-eee51f135c7b service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:28.416712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1264/2527] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:31.250173 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:31.379269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ad6f-6ecb-497b-ab68-a37fe8b5fd2c service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:31.386618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:31.387622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.388198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.388706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] PATH_INFO: `/v3/users/64b80bd1edb74f6db0dcbdfe966b10db` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.391175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b80bd1edb74f6db0dcbdfe966b10db)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:31.395494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:31.492131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b80bd1edb74f6db0dcbdfe966b10db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:31.492852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b80bd1edb74f6db0dcbdfe966b10db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:31.503683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b80bd1edb74f6db0dcbdfe966b10db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:31.531102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b80bd1edb74f6db0dcbdfe966b10db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:31.532190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b80bd1edb74f6db0dcbdfe966b10db'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:31.558052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1264/2528] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:31 2022] DELETE /identity/v3/users/64b80bd1edb74f6db0dcbdfe966b10db => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:31.565744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.566656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.566656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.602566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:31.604582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:31.769030 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1265/2529] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:31.785800 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:31.788369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:31.917669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3bb825-b749-4061-b8d7-586b39d8c81e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:31.922514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:31.923549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.925627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.926259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.939059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:31.942553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:31.960401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1265/2530] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:32.455413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:32.457793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:32.474338 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700248fd-b8b2-4353-9bdf-53bda2460136 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:32.482136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:32.483012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:32.483299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:32.483573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] PATH_INFO: `/v3/projects/4203814ad37f43f1b26f89885f5a02ba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:32.484798 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4203814ad37f43f1b26f89885f5a02ba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:32.502298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:32.568944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4203814ad37f43f1b26f89885f5a02ba', 'request_id': 'req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.581385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4203814ad37f43f1b26f89885f5a02ba', 'request_id': 'req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.587363 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:32.595099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4203814ad37f43f1b26f89885f5a02ba', 'request_id': 'req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.617051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] The token cache is being invalidate because project 4203814ad37f43f1b26f89885f5a02ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:32.617424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.618139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.619979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1266/2531] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:32 2022] DELETE /identity/v3/projects/4203814ad37f43f1b26f89885f5a02ba => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:32.624604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:32.633524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:32.639666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bffea-d37a-4a2e-88f3-81e7afce4800 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:32.711421 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:32.712171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:32.712453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:32.712730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] PATH_INFO: `/v3/users/1b655a134b954185aa268275adc9f2cb` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:32.713909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b655a134b954185aa268275adc9f2cb)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:32.715896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:32.778112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b655a134b954185aa268275adc9f2cb'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.779411 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b655a134b954185aa268275adc9f2cb'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.796325 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b655a134b954185aa268275adc9f2cb'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.812331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b655a134b954185aa268275adc9f2cb'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.813274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-69383f0d-f464-473c-b5da-a9c444afc091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b655a134b954185aa268275adc9f2cb'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:32.838821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1266/2532] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:32 2022] DELETE /identity/v3/users/1b655a134b954185aa268275adc9f2cb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:33.207639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.209628 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:33.224645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c8ebb-cb78-4fea-abd0-1ad8f2f28548 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.326354 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.328510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.328510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.328911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] PATH_INFO: `/v3/projects/93a1c03176734cf2ac61fef9a0cb39f8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.330601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a1c03176734cf2ac61fef9a0cb39f8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:33.333993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:33.399234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a1c03176734cf2ac61fef9a0cb39f8', 'request_id': 'req-0761a792-796b-4097-9df7-0db412d91f4d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:33.408357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a1c03176734cf2ac61fef9a0cb39f8', 'request_id': 'req-0761a792-796b-4097-9df7-0db412d91f4d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:33.417383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a1c03176734cf2ac61fef9a0cb39f8', 'request_id': 'req-0761a792-796b-4097-9df7-0db412d91f4d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:33.431004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] The token cache is being invalidate because project 93a1c03176734cf2ac61fef9a0cb39f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:33.431317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:33.432726 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0761a792-796b-4097-9df7-0db412d91f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:33.433750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1267/2533] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:33 2022] DELETE /identity/v3/projects/93a1c03176734cf2ac61fef9a0cb39f8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:33.438993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.439283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.439563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.468437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:33.470480 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:33.575942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1267/2534] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:33.587685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.588302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.588782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.617089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:33.618359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:33.629513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1268/2535] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:33.633557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.649285 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bb0ee-a197-43be-9134-dd80393e5e3e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.657277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.658254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.659342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.659853 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.661095 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:33.661608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:33.664695 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:33.682209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1268/2536] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:33.686862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.698706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bef6b-012d-44b8-93e0-98b6d92abd65 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.704107 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.704727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.704999 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.705268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.706613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:33.708922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:33.751717 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1269/2537] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:33 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:33.757028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.770189 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7fab4-0df9-442d-bec1-cdf7b8460bc9 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.780524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:33.851844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.852844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.853314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.853761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.854806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:33.858585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:33.891271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1269/2538] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:33 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:33.895647 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.906774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a7ad5-15c2-4667-a863-c66954cad284 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.911914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.912529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.912809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.913081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.913724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:33.915388 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:33.927000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1270/2539] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:33.931361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.942738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd916d1-7fe9-4307-8e13-48fd9e18ddbd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.947797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e94b-d3c5-42b1-897d-47fc82d0393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.948442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e94b-d3c5-42b1-897d-47fc82d0393f admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.948721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e94b-d3c5-42b1-897d-47fc82d0393f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.948993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e94b-d3c5-42b1-897d-47fc82d0393f admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:33.949642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793e94b-d3c5-42b1-897d-47fc82d0393f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:33.951433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793e94b-d3c5-42b1-897d-47fc82d0393f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:33.964677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1270/2540] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:33.973753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:33.989986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8b37c-d723-4d7a-ab16-23d6c39ec40d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:33.996875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:33.997827 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:33.998272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:33.998689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] PATH_INFO: `/v3/projects/b9e461c86f0242aba7948cb596d319d9/users/821ee239d8ce43aebb4385fb365e1cf6/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.023596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e461c86f0242aba7948cb596d319d9, user_id=821ee239d8ce43aebb4385fb365e1cf6, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.028567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.046599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1271/2541] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:33 2022] PUT /identity/v3/projects/b9e461c86f0242aba7948cb596d319d9/users/821ee239d8ce43aebb4385fb365e1cf6/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:34.058021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.058414 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.059306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.105075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] MFA Rules not processed for user `821ee239d8ce43aebb4385fb365e1cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:34.105075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:34.183498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1271/2542] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:34.188203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:34.206361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:34.273762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1547da5-5c11-43f7-8b95-5237af3756f6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:34.278498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:34.279150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.279430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.279705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.280350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.282268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.314299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1272/2543] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:34 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:34.318505 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:34.330079 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41761d-72cf-4b32-8e4f-526ffae56276 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:34.397917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:34.398539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.398819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.399170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.399841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.401972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.434734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1272/2544] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:34 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:34.437254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:34.449795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114a16-0680-4fd7-93a3-269c8d62774a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:34.457178 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:34.458098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.458559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.459061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.460119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.461790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.475309 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1273/2545] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:34.484562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:34.495969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0571285-079b-423a-86cf-41a26bf428a5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:34.501991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:34.504563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.504850 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.505134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] PATH_INFO: `/v3/projects/c631c177c10c4558a543283bda7b28fe/users/308407f1a71548bfbceb0df1a718c87d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.510743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:34.551685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c631c177c10c4558a543283bda7b28fe, user_id=308407f1a71548bfbceb0df1a718c87d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.561214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.611007 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1273/2546] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:34 2022] PUT /identity/v3/projects/c631c177c10c4558a543283bda7b28fe/users/308407f1a71548bfbceb0df1a718c87d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:34.612213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:34.631504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dfd96-4f93-449c-92c2-217d1f6f4c18 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:34.753411 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c3204-e916-4a2e-81f4-0d3c82e39548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:34.754204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c3204-e916-4a2e-81f4-0d3c82e39548 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.754757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c3204-e916-4a2e-81f4-0d3c82e39548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.755168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c3204-e916-4a2e-81f4-0d3c82e39548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.756989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c3204-e916-4a2e-81f4-0d3c82e39548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.758965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c3204-e916-4a2e-81f4-0d3c82e39548 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.771862 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1274/2547] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:34.777387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:34.795225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bc8d-4eeb-4976-9dc5-62c58a3a68bf admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:34.803758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:34.804589 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.804952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.805346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] PATH_INFO: `/v3/projects/c631c177c10c4558a543283bda7b28fe/users/308407f1a71548bfbceb0df1a718c87d/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.808598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c631c177c10c4558a543283bda7b28fe, user_id=308407f1a71548bfbceb0df1a718c87d, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:34.813556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:34.829488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1274/2548] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:34 2022] PUT /identity/v3/projects/c631c177c10c4558a543283bda7b28fe/users/308407f1a71548bfbceb0df1a718c87d/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:34.841812 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:34.842331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:34.842903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:34.884347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] MFA Rules not processed for user `308407f1a71548bfbceb0df1a718c87d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:34.885926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:35.017916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1275/2549] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:35.033241 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:35.035970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:35.135139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8f6c2-a05c-4028-be57-b9e7e37e98b4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:35.140866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:35.141602 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:35.141979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:35.142343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:35.160528 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:35.162649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:35.177441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1275/2550] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:35.716976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:35.730563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:35.760183 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:35.762809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:35.795344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7602-5575-48a2-b089-d556d7e18976 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:35.804360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:35.805640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:35.806450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:35.807098 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:35.813455 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:35.907573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693478a3-c528-4dad-84b9-e183fd52b971 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:35.925333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:35.926004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:35.926282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:35.926548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] PATH_INFO: `/v3/users/821ee239d8ce43aebb4385fb365e1cf6` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:35.928382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] RBAC: Authorizing `identity:delete_user(user_id=821ee239d8ce43aebb4385fb365e1cf6)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:35.930337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:35.969390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:35.969390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe4503-9a69-4620-87c4-cccd1e88f9f9 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:35.983166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1276/2551] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:36.103074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821ee239d8ce43aebb4385fb365e1cf6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:36.103790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821ee239d8ce43aebb4385fb365e1cf6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:36.107367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821ee239d8ce43aebb4385fb365e1cf6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:36.127158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821ee239d8ce43aebb4385fb365e1cf6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:36.153460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821ee239d8ce43aebb4385fb365e1cf6'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:36.208071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1276/2552] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:35 2022] DELETE /identity/v3/users/821ee239d8ce43aebb4385fb365e1cf6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:36.214577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:36.215975 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:36.216723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:36.314535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:36.316789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:36.549628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1277/2553] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:36.561997 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:36.563885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:36.670287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a67471-535c-44f2-a488-5531c457b27e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:36.675211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:36.675542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:36.675826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:36.676103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:36.688811 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:36.692132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:36.704217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1277/2554] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:37.110807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:37.113154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:37.127521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb5632-9920-4761-9479-99f0ced07a21 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:37.133680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:37.134441 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:37.135002 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:37.135485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] PATH_INFO: `/v3/projects/b9e461c86f0242aba7948cb596d319d9` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:37.137019 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e461c86f0242aba7948cb596d319d9)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:37.139137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:37.154654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:37.228786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e461c86f0242aba7948cb596d319d9', 'request_id': 'req-c63c532a-da31-4eee-a67d-68f55a48549b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.239182 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e461c86f0242aba7948cb596d319d9', 'request_id': 'req-c63c532a-da31-4eee-a67d-68f55a48549b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.249197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e461c86f0242aba7948cb596d319d9', 'request_id': 'req-c63c532a-da31-4eee-a67d-68f55a48549b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.267724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] The token cache is being invalidate because project b9e461c86f0242aba7948cb596d319d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:37.268629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.269571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.274903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1278/2555] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:37 2022] DELETE /identity/v3/projects/b9e461c86f0242aba7948cb596d319d9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:37.277773 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:37.296749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553a69d-e659-479a-a484-a0f27e9441e3 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:37.337511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:37.385158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:37.386288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:37.386576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:37.386867 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] PATH_INFO: `/v3/users/308407f1a71548bfbceb0df1a718c87d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:37.388120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308407f1a71548bfbceb0df1a718c87d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:37.390084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:37.457086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308407f1a71548bfbceb0df1a718c87d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.457894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308407f1a71548bfbceb0df1a718c87d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.458857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308407f1a71548bfbceb0df1a718c87d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.472022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308407f1a71548bfbceb0df1a718c87d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.492296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-df24274e-4705-4fea-8704-5e462e43e099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308407f1a71548bfbceb0df1a718c87d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:37.517136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1278/2556] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:37 2022] DELETE /identity/v3/users/308407f1a71548bfbceb0df1a718c87d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:37.944401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:37.946425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:37.972449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c532a-da31-4eee-a67d-68f55a48549b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.048647 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.049942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.050249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.050632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] PATH_INFO: `/v3/projects/c631c177c10c4558a543283bda7b28fe` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.052624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c631c177c10c4558a543283bda7b28fe)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.056368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.126675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c631c177c10c4558a543283bda7b28fe', 'request_id': 'req-03f410e0-d0bc-494e-87a2-17806655ab43'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:38.134205 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c631c177c10c4558a543283bda7b28fe', 'request_id': 'req-03f410e0-d0bc-494e-87a2-17806655ab43'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:38.144951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c631c177c10c4558a543283bda7b28fe', 'request_id': 'req-03f410e0-d0bc-494e-87a2-17806655ab43'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:38.161913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] The token cache is being invalidate because project c631c177c10c4558a543283bda7b28fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:38.162284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:38.163569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03f410e0-d0bc-494e-87a2-17806655ab43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:38.165487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1279/2557] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:37 2022] DELETE /identity/v3/projects/c631c177c10c4558a543283bda7b28fe => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:38.171268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.171550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.171740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.195541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:38.196761 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:38.274215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1279/2558] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:38.282903 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.283192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.283466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.306886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:38.308832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:38.319278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1280/2559] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:38.324374 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.339208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfac73d-368a-4cf3-b165-b998d9c5bf8c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.355569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.356211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.356527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.356891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.357916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:38.358227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.359901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.377950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1280/2560] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:06:38.382050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.392009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9dc08-e34b-4615-9d5a-68bc2d2be929 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.397580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.398186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.398466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.398731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.399412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.401349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.427991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:38.439729 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1281/2561] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:38 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:38.444563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.456493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e86c3a-7613-47e5-9b1c-c4ab9adc5739 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.499378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:38.544987 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.545724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.546091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.546558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.547594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.550677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.582404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1281/2562] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:38 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:38.586997 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.599779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7dc33-8024-476e-9399-f8598b51d939 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.604551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.605180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.605662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.605946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.606714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.608357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.622486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1282/2563] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:38.627510 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.641760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe2c2e-500b-4893-960b-820dbffba31f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.648759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fea981-2b6f-40d6-8edb-e6af4266286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.650413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fea981-2b6f-40d6-8edb-e6af4266286d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.650413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fea981-2b6f-40d6-8edb-e6af4266286d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.650413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fea981-2b6f-40d6-8edb-e6af4266286d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.651875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fea981-2b6f-40d6-8edb-e6af4266286d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.652660 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fea981-2b6f-40d6-8edb-e6af4266286d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.665689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1282/2564] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:38.671716 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.692177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d776db-fde6-49a4-9a44-e792f977b688 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:38.695265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:38.695265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.695265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.695743 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] PATH_INFO: `/v3/projects/a8ad1064951a42f2aeb5539e8151bd3e/users/8e397cea6df84de6aa4b1e18d5727f54/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.718296 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ad1064951a42f2aeb5539e8151bd3e, user_id=8e397cea6df84de6aa4b1e18d5727f54, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:38.721901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:38.743569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1283/2565] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:38 2022] PUT /identity/v3/projects/a8ad1064951a42f2aeb5539e8151bd3e/users/8e397cea6df84de6aa4b1e18d5727f54/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:38.760059 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:38.760367 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:38.760640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:38.793782 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] MFA Rules not processed for user `8e397cea6df84de6aa4b1e18d5727f54`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:38.795413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:38.887190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1283/2566] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:38.891813 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:38.998096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f303a84-fb0b-4312-b7bd-cce974d79148 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.005081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.005712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.005993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.006269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.006979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.008909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.049125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1284/2567] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:38 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:39.053521 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:39.072080 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5103a-bc02-4486-9f18-87d143be2cc3 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.143213 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.143855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.144124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.144396 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.145069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.147040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.185163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1284/2568] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:39 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:39.189257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:39.206339 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4409b0-aa1f-47c6-a283-c4dd3952afc0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.214395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.215153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.215495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.215841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.216571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.218337 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.231105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1285/2569] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:39.235546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:39.248012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c2d65-4c99-44fb-b9f8-552a739d33cc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.253904 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.254642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.254991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.255250 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] PATH_INFO: `/v3/projects/f0b1e951018048558426ed03c5c254ba/users/8cee2db4b6b34f11882cc0ac8373694e/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.272336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b1e951018048558426ed03c5c254ba, user_id=8cee2db4b6b34f11882cc0ac8373694e, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.275111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.302383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1285/2570] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:39 2022] PUT /identity/v3/projects/f0b1e951018048558426ed03c5c254ba/users/8cee2db4b6b34f11882cc0ac8373694e/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:39.306535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:39.320433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e7e3f-587a-451a-aa6a-fe10e3fa1462 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.442616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31a0fe-8d3e-4944-840c-95c7a5c70825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.444438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31a0fe-8d3e-4944-840c-95c7a5c70825 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.444438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31a0fe-8d3e-4944-840c-95c7a5c70825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.444987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31a0fe-8d3e-4944-840c-95c7a5c70825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.458895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31a0fe-8d3e-4944-840c-95c7a5c70825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.458895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31a0fe-8d3e-4944-840c-95c7a5c70825 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.472788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1286/2571] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:39.476176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:39.492835 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1195-e815-4450-83ee-a4443e7d5f89 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.502930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.503636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.503996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.504335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] PATH_INFO: `/v3/projects/f0b1e951018048558426ed03c5c254ba/users/8cee2db4b6b34f11882cc0ac8373694e/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.506815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b1e951018048558426ed03c5c254ba, user_id=8cee2db4b6b34f11882cc0ac8373694e, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.509562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.514242 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:39.525977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1286/2572] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:39 2022] PUT /identity/v3/projects/f0b1e951018048558426ed03c5c254ba/users/8cee2db4b6b34f11882cc0ac8373694e/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:39.546739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.547084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.547374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.582709 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] MFA Rules not processed for user `8cee2db4b6b34f11882cc0ac8373694e`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:39.584140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:39.608767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:39.685225 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1287/2573] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:39.698976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:39.701417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:39.777857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d5265-ebe9-47af-850e-f793cf274d24 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:39.782754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:39.783443 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:39.783795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:39.784185 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:39.856259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:39.857821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:39.870600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1287/2574] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:41.978435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:41.978435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:42.084352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3c75-a7d3-486f-8a3c-3ad31b461353 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:42.088764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:42.089649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:42.089649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:42.090256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:42.092463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:42.201548 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:42.206374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e07a5c-8ce5-4e9b-af5d-7a007bc5fbd7 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:42.225440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1288/2575] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:44.631162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:44.634105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:44.679576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394be471-6017-4728-9e84-6687350cc51e service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:44.695700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:44.706390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:44.706390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:44.706909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:44.719310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:44.749384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:44.750923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:44.756233 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:44.757151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:44.757435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:44.766936 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1288/2576] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:44.821372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] MFA Rules not processed for user `9979be70f604408d9b78050e3ba52f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:44.826586 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:44.991065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1289/2577] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:45.943590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:45.965131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87e7c3-9833-4b3c-b797-de90844ebc01 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:45.987828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:45.988464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:45.988753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:45.989233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:46.018504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:46.022297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:46.043053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1289/2578] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:48.019262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:48.021305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:48.076270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18563e0-31e2-4290-86a8-27eee99a5f77 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:48.086090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:48.087698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:48.088214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:48.090749 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] PATH_INFO: `/v3/users/4bbd5a38a6ce472790128a37f62d472b` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:48.093167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bbd5a38a6ce472790128a37f62d472b)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:48.095720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:48.134504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:48.185739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bbd5a38a6ce472790128a37f62d472b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.186940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bbd5a38a6ce472790128a37f62d472b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.202578 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bbd5a38a6ce472790128a37f62d472b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.229060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bbd5a38a6ce472790128a37f62d472b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.230318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fb089696-e7b9-4a72-8d99-1eec90075878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bbd5a38a6ce472790128a37f62d472b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.275254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1290/2579] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:48 2022] DELETE /identity/v3/users/4bbd5a38a6ce472790128a37f62d472b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:48.652251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:48.667631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375f553-ff78-4b38-b2f1-2c8092ff9a0e service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:48.790961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:48.791701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:48.792017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:48.792519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] PATH_INFO: `/v3/projects/3f1770ed75fd48d098bb1efc839ebc0d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:48.794135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1770ed75fd48d098bb1efc839ebc0d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:48.795972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:48.867276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:48.902384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1770ed75fd48d098bb1efc839ebc0d', 'request_id': 'req-11eafe1d-87b5-4090-97c0-19b9bbdebeac'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.917944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1770ed75fd48d098bb1efc839ebc0d', 'request_id': 'req-11eafe1d-87b5-4090-97c0-19b9bbdebeac'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.935476 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1770ed75fd48d098bb1efc839ebc0d', 'request_id': 'req-11eafe1d-87b5-4090-97c0-19b9bbdebeac'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.957535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] The token cache is being invalidate because project 3f1770ed75fd48d098bb1efc839ebc0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:48.958033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.959387 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-11eafe1d-87b5-4090-97c0-19b9bbdebeac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:48.963146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1290/2580] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:48 2022] DELETE /identity/v3/projects/3f1770ed75fd48d098bb1efc839ebc0d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:48.968876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:48.969683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:48.970402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.011894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:49.013904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:49.151318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1291/2581] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:49.154076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.154754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.155752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.220315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:49.226464 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:49.241039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1291/2582] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:49.245106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:49.256245 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e9169-afba-456e-8cf5-b0a8e1409838 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:49.262745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:49.263517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.264014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.264454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.265382 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:49.265852 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:49.268956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:49.296985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1292/2583] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:49.300436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:49.320598 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe300a-5a5d-405f-bf78-dc8b62142a57 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:49.328086 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:49.329696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.330417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.331787 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.333220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:49.337436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:49.396390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1292/2584] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:49 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:49.405605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:49.417718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caa0ed-8123-4142-953d-b1f1ed427675 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:49.547653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:49.548499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.548790 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.549075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.552038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:49.554127 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:49.585388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1293/2585] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:49 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:49.593251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:49.619526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f9c4-3526-4b63-bfb3-df5cc1ec6dc5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:49.628597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:49.629584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.630656 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.631376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.632815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:49.635814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:49.691851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:49.692050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1293/2586] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:49.709230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:49.751956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaf229-6a67-4337-97c5-908dca5ef4b1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:49.779953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9b0bd-9314-4f5f-9e8b-54c77e7a4121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:49.782340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9b0bd-9314-4f5f-9e8b-54c77e7a4121 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.783142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9b0bd-9314-4f5f-9e8b-54c77e7a4121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.783753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9b0bd-9314-4f5f-9e8b-54c77e7a4121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.786966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9b0bd-9314-4f5f-9e8b-54c77e7a4121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:49.791192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9b0bd-9314-4f5f-9e8b-54c77e7a4121 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:49.837273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1294/2587] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:49.841684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:49.867541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d09a6a-8e4f-468c-abb2-28200df49493 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:49.888271 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:49.889930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.892074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.893583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] PATH_INFO: `/v3/projects/0b42d353672549d99542c699f7fc8a53/users/2b24cfa819294829aa95e0c74c476072/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:49.945655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b42d353672549d99542c699f7fc8a53, user_id=2b24cfa819294829aa95e0c74c476072, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:49.948682 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:49.989137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1294/2588] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:49 2022] PUT /identity/v3/projects/0b42d353672549d99542c699f7fc8a53/users/2b24cfa819294829aa95e0c74c476072/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:49.994599 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:49.995739 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:49.996661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:50.070355 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] MFA Rules not processed for user `2b24cfa819294829aa95e0c74c476072`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:50.072913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:50.214289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1295/2589] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:50.227896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:50.232666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:50.491707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cee7f-4298-4e7d-810f-5e2820741699 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:50.498232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:50.499376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:50.499873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:50.500607 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:50.512819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:50.515145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:50.536432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1295/2590] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:51.844452 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:51.846808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:51.907963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:51.914028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:51.993304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da688b14-8fef-48ea-9a0c-1150d19d215d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.000900 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.001766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.002300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.003445 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:52.003930 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] PATH_INFO: `/v3/users/28195a63990344edb6ea31f1f4463a82` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.006144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28195a63990344edb6ea31f1f4463a82)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.010088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.066901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5301d-ac5d-47fb-b584-f831a1dffa38 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.087225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.087963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.088273 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.088525 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] PATH_INFO: `/v3/users/8e397cea6df84de6aa4b1e18d5727f54` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.090075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e397cea6df84de6aa4b1e18d5727f54)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.093259 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.149137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28195a63990344edb6ea31f1f4463a82'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.150517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28195a63990344edb6ea31f1f4463a82'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.173228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28195a63990344edb6ea31f1f4463a82'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.182635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e397cea6df84de6aa4b1e18d5727f54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.183842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e397cea6df84de6aa4b1e18d5727f54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.185028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e397cea6df84de6aa4b1e18d5727f54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.195300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28195a63990344edb6ea31f1f4463a82'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.197105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-03b4abc0-6a5e-49b9-b00c-947ff8bb882f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28195a63990344edb6ea31f1f4463a82'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.199426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e397cea6df84de6aa4b1e18d5727f54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.217329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:52.227037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e397cea6df84de6aa4b1e18d5727f54'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.228563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1296/2591] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:51 2022] DELETE /identity/v3/users/28195a63990344edb6ea31f1f4463a82 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:52.258043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1296/2592] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:51 2022] DELETE /identity/v3/users/8e397cea6df84de6aa4b1e18d5727f54 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:52.263165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.263761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.264396 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.296572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:52.298160 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:52.424137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1297/2593] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:52.443192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:52.445029 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:52.525055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c002-aa19-4fc6-b134-efa01829943e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.529277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465a893-1383-44d5-b47b-2ef73c4a6db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.530623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465a893-1383-44d5-b47b-2ef73c4a6db1 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.531216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465a893-1383-44d5-b47b-2ef73c4a6db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.532538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465a893-1383-44d5-b47b-2ef73c4a6db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.548189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465a893-1383-44d5-b47b-2ef73c4a6db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.551509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465a893-1383-44d5-b47b-2ef73c4a6db1 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.563027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1297/2594] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:52.801910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:52.804514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:52.825049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7a098-0d59-4b4d-a596-9a79166b9cb3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.833605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.834402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.834793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.835198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] PATH_INFO: `/v3/projects/993e0f8cdcb744279b2987d0d8f321ae` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.837754 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993e0f8cdcb744279b2987d0d8f321ae)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.840384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.956635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993e0f8cdcb744279b2987d0d8f321ae', 'request_id': 'req-ba08af7c-886c-4151-aa44-3cfcbd0eb107'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.980961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993e0f8cdcb744279b2987d0d8f321ae', 'request_id': 'req-ba08af7c-886c-4151-aa44-3cfcbd0eb107'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.000643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993e0f8cdcb744279b2987d0d8f321ae', 'request_id': 'req-ba08af7c-886c-4151-aa44-3cfcbd0eb107'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.024420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] The token cache is being invalidate because project 993e0f8cdcb744279b2987d0d8f321ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:53.024801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.033270 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.037052 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1298/2595] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:52 2022] DELETE /identity/v3/projects/993e0f8cdcb744279b2987d0d8f321ae => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:53.044597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5de46-c818-4bde-bed2-7ad20c5828ed None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.045490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5de46-c818-4bde-bed2-7ad20c5828ed None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.045490 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5de46-c818-4bde-bed2-7ad20c5828ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.078573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-24c5de46-c818-4bde-bed2-7ad20c5828ed None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:53.080025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-24c5de46-c818-4bde-bed2-7ad20c5828ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.184866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:53.212160 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1298/2596] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:53.222886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.224631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.225148 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.226584 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.226947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.388590 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:53.389751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.406266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08af7c-886c-4151-aa44-3cfcbd0eb107 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.413213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.413985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.414319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.414604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] PATH_INFO: `/v3/projects/a8ad1064951a42f2aeb5539e8151bd3e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.416034 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ad1064951a42f2aeb5539e8151bd3e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.418170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.423110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1299/2597] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:53.431790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.442998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:53.446892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25be1bc-cd93-49aa-9c4e-c3fde517739e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.458167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.459252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.459826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.460217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.461245 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:53.461776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.463754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.490769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1300/2598] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:53.495711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.506208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ad1064951a42f2aeb5539e8151bd3e', 'request_id': 'req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.510351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc77ff-0813-4fdb-824f-a9bb84094aef admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.520288 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ad1064951a42f2aeb5539e8151bd3e', 'request_id': 'req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.531787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ad1064951a42f2aeb5539e8151bd3e', 'request_id': 'req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.560242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] The token cache is being invalidate because project a8ad1064951a42f2aeb5539e8151bd3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:53.564214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.565667 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.569141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1299/2599] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:53 2022] DELETE /identity/v3/projects/a8ad1064951a42f2aeb5539e8151bd3e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:53.577983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.581899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.667077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.673088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.673591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.674513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.675794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.680342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.716767 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3893-9e90-45fa-b93b-d9e9bdc085bc admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.724811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.725520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.726151 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.726907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] PATH_INFO: `/v3/users/8cee2db4b6b34f11882cc0ac8373694e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.728917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cee2db4b6b34f11882cc0ac8373694e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.731635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.770526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1301/2600] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:53 2022] POST /identity/v3/projects => generated 403 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:53.782813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.787998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.853793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cee2db4b6b34f11882cc0ac8373694e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.855050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cee2db4b6b34f11882cc0ac8373694e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.874908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cee2db4b6b34f11882cc0ac8373694e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.897908 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cee2db4b6b34f11882cc0ac8373694e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.899706 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cee2db4b6b34f11882cc0ac8373694e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.925008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353d343-730e-4752-a936-4ad2262665b3 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.934420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1300/2601] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:53 2022] DELETE /identity/v3/users/8cee2db4b6b34f11882cc0ac8373694e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:53.938066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.939182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.939623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.940500 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.941799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.944409 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.964633 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:54.007428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1302/2602] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:53 2022] POST /identity/v3/users => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:54.015857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.017600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.081100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:54.179559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aa590-ff5e-4d61-9c90-1a26e0f508d0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.187621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.188612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.189073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.189533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.191041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.193996 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.224409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1301/2603] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:54.229093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.242685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961036f1-ca65-45af-a462-b7c06267851a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.249108 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.250667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.250667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.250667 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.251503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.254137 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.273860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1303/2604] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:54.275135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.287512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a9dd-e7eb-4166-9c14-e5ea8ac91772 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.294260 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c181e3a-7091-450b-ac55-6ecc92a14970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.295118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c181e3a-7091-450b-ac55-6ecc92a14970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.295736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c181e3a-7091-450b-ac55-6ecc92a14970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.295935 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c181e3a-7091-450b-ac55-6ecc92a14970 admin admin] PATH_INFO: `/v3/projects/061272abc61045ff95cda9786b9bcf0a/users/168f4f7f972548e6b7de4e43073e469c/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.328073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c181e3a-7091-450b-ac55-6ecc92a14970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061272abc61045ff95cda9786b9bcf0a, user_id=168f4f7f972548e6b7de4e43073e469c, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.328073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c181e3a-7091-450b-ac55-6ecc92a14970 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.337573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.341111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.350881 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1302/2605] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:54 2022] PUT /identity/v3/projects/061272abc61045ff95cda9786b9bcf0a/users/168f4f7f972548e6b7de4e43073e469c/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:54.357015 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.357342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.357740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.361905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d614b79-144c-47ed-8243-2dffe5be78dc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.385911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:54.387851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.484635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1303/2606] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:54.486215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.488033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.488815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.491519 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] PATH_INFO: `/v3/users/2b24cfa819294829aa95e0c74c476072` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.493698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b24cfa819294829aa95e0c74c476072)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.497417 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.502178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.506364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.595796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b24cfa819294829aa95e0c74c476072'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.596976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b24cfa819294829aa95e0c74c476072'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.598494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b24cfa819294829aa95e0c74c476072'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.616586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b24cfa819294829aa95e0c74c476072'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.650531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e1aec144-fc28-4916-b206-0a852605f34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b24cfa819294829aa95e0c74c476072'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.662056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404724dc-f6e9-4b6a-9a63-81167f26baa9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.677184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.678103 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1304/2607] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:54 2022] DELETE /identity/v3/users/2b24cfa819294829aa95e0c74c476072 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:54.678264 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.678763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.679298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.682564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.684762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.685253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.685683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.703981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.706358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.724837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:54.726430 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.730950 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1304/2608] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:54.753765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.755255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.769807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c81f72-37c5-48e0-ab64-815a4346a5d0 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.814684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:54.865310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1305/2609] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:54.881303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.883672 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.890771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:54.908527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.909427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.909713 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.910006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] PATH_INFO: `/v3/projects/f0b1e951018048558426ed03c5c254ba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.911784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b1e951018048558426ed03c5c254ba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.913761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:55.023493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c98e0d-a9af-455d-b4e0-33bd39758881 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:55.029536 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fa37d-3254-4c64-8a59-645bd90eb39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:55.030663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fa37d-3254-4c64-8a59-645bd90eb39e service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.031019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fa37d-3254-4c64-8a59-645bd90eb39e service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.031383 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fa37d-3254-4c64-8a59-645bd90eb39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.045249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fa37d-3254-4c64-8a59-645bd90eb39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:55.046937 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fa37d-3254-4c64-8a59-645bd90eb39e service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:55.048283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b1e951018048558426ed03c5c254ba', 'request_id': 'req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.058174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b1e951018048558426ed03c5c254ba', 'request_id': 'req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.063199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1306/2610] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:55.068104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b1e951018048558426ed03c5c254ba', 'request_id': 'req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.090694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] The token cache is being invalidate because project f0b1e951018048558426ed03c5c254ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:55.091242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.092776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4565ca07-3d2d-4e7c-8a9c-bcfa316b4bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.097050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1305/2611] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:54 2022] DELETE /identity/v3/projects/f0b1e951018048558426ed03c5c254ba => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:55.112891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.113341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.113513 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.164382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:55.167229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:55.382004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1307/2612] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:55.401595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.402134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.402758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.456551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:55.460041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:55.497077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1306/2613] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:55.505428 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:55.541401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae391c9-dbe1-4df7-90a0-f100bcf09c98 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:55.563541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:55.563541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.564452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.564917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.566489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:55.567017 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:55.569262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:55.594911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1308/2614] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:55.600989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:55.635615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d87a44-4e15-4008-9866-927bc4faabfa None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:55.646777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:55.647469 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.647871 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.649332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.649803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:55.654030 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:55.674529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:55.677262 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:55.707424 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106e253-60f7-4d08-91e1-6c26a9d976fb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:55.718648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:55.720081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.720779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.722643 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] PATH_INFO: `/v3/projects/0b42d353672549d99542c699f7fc8a53` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.725640 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b42d353672549d99542c699f7fc8a53)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:55.736433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1307/2615] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:55 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:55.741911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:55.743991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:55.840364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b42d353672549d99542c699f7fc8a53', 'request_id': 'req-ede0522f-0b44-4e93-a815-88d57c2c70ae'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.848960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b42d353672549d99542c699f7fc8a53', 'request_id': 'req-ede0522f-0b44-4e93-a815-88d57c2c70ae'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.864868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b42d353672549d99542c699f7fc8a53', 'request_id': 'req-ede0522f-0b44-4e93-a815-88d57c2c70ae'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.890392 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba4fca-5aed-473d-9731-14f74f28c8f3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:55.909324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] The token cache is being invalidate because project 0b42d353672549d99542c699f7fc8a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:55.910099 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.911106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-ede0522f-0b44-4e93-a815-88d57c2c70ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.916119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1309/2616] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:55 2022] DELETE /identity/v3/projects/0b42d353672549d99542c699f7fc8a53 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:55.926410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:55.927261 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:55.927882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:55.966521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:55.972221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:55.976485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:55.998326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:56.061097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df35bfd-f521-4dbb-9b4f-84347774760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.061955 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df35bfd-f521-4dbb-9b4f-84347774760d admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.062106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df35bfd-f521-4dbb-9b4f-84347774760d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.062259 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df35bfd-f521-4dbb-9b4f-84347774760d admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.063096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df35bfd-f521-4dbb-9b4f-84347774760d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.066190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df35bfd-f521-4dbb-9b4f-84347774760d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.101350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1308/2617] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:55 2022] POST /identity/v3/users => generated 286 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:56.101350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.101350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.101350 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.139993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] MFA Rules not processed for user `168f4f7f972548e6b7de4e43073e469c`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:56.141318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.146958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1310/2618] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:56.146958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.160419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.185601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75003b00-e1d4-4b56-ac15-7214e0b53cd9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.193716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf95e3-a419-45d1-9b38-16b1ec312c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.194750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf95e3-a419-45d1-9b38-16b1ec312c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.195356 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf95e3-a419-45d1-9b38-16b1ec312c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.195767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf95e3-a419-45d1-9b38-16b1ec312c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.197025 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf95e3-a419-45d1-9b38-16b1ec312c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.201477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf95e3-a419-45d1-9b38-16b1ec312c7f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.225351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1311/2619] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:56.231419 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.232170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.232622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.254148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1309/2620] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:56.254913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.266763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4e0a1-97f5-4145-a3be-a306ec83f5b1 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.271922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:56.272884 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.273592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.273711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.273970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.274352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.275114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.276857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.293265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1312/2621] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:56.293476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1310/2622] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:56.293610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.300263 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.301649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.331687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e42d4-8129-460e-b212-0b81e09cb2b9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.333284 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde1d30-28b1-41db-b8b7-a9a1a72e2d4a admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.341966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.342641 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.343147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.343428 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.344468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:56.344967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.345569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.347066 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.347629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.351114 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.351732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.352654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.354917 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.371483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1311/2623] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:56.373384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.404093 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1313/2624] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:56.404934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6dbe1b-396c-4084-a31b-daa1505e3348 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.407534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.511703 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d5e9e-c375-4b36-a6b9-89cc6f322620 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.512863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.513994 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.514516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.515679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] PATH_INFO: `/v3/projects/b58ee7c3a21941acad15b0d3dffc504e/users/6fd73a2f6bcd429a98e0a4ed3085e0ac/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.525220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.526174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.527133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.527289 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.528313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.532110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.551642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58ee7c3a21941acad15b0d3dffc504e, user_id=6fd73a2f6bcd429a98e0a4ed3085e0ac, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.555446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.587299 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1312/2626] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:56 2022] PUT /identity/v3/projects/b58ee7c3a21941acad15b0d3dffc504e/users/6fd73a2f6bcd429a98e0a4ed3085e0ac/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:56.587488 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1314/2626] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:56.587582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.590052 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.626041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:56.693492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff22c87-0bfd-4687-ba06-b3230230f910 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.699580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770270a-8a91-40a4-a1e4-f61dc988e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.700266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770270a-8a91-40a4-a1e4-f61dc988e9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.700555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770270a-8a91-40a4-a1e4-f61dc988e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.700858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770270a-8a91-40a4-a1e4-f61dc988e9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.701626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770270a-8a91-40a4-a1e4-f61dc988e9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.702927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1d824-9839-482f-842d-e53847fcb332 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.703963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770270a-8a91-40a4-a1e4-f61dc988e9f8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.709142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.709868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.710246 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.710786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.711715 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.714377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.714632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:56.734260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1315/2627] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:56.734260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31501b40-d707-419c-a241-537aee62ead5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.734260 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31501b40-d707-419c-a241-537aee62ead5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.737729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31501b40-d707-419c-a241-537aee62ead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.746660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1313/2628] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/users => generated 308 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:56.748035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.762687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660e0a4-61d9-4ef3-8385-3e0ff190e184 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.767631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.768907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.769198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.769451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.770191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.772192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.781110 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-31501b40-d707-419c-a241-537aee62ead5 None None] MFA Rules not processed for user `6fd73a2f6bcd429a98e0a4ed3085e0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:56.782344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-31501b40-d707-419c-a241-537aee62ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.793293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1314/2629] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:56.793293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.809593 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ba374-0771-40ea-bf3c-2fb7228f4d13 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.818424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.819430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.819944 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.820461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.821649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.824791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.840820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1315/2630] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:56.841701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.853659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50a67a-e8e9-4038-bf26-6aa043f3360e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.861017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.862553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.863041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.863665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.864910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.867322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.882073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1316/2631] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:56.883462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.897143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafa5f-5cf2-44a6-97aa-2eb6be7dfdaa admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.900474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1316/2632] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:56.902236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31501b40-d707-419c-a241-537aee62ead5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.913013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.913459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.913917 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.914391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] PATH_INFO: `/v3/projects/cd67f1d0479e4f0893abfc00390db9a0/users/3920e0382d4049359ed208f796d8b9c8/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.926133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31501b40-d707-419c-a241-537aee62ead5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.932489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.933366 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.933814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.934190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] PATH_INFO: `/v3/projects/a627f25c25864076a0e3cb5f77cf755d/users/18c75adf08e34786a3bd4a847e4a4c4e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.940168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd67f1d0479e4f0893abfc00390db9a0, user_id=3920e0382d4049359ed208f796d8b9c8, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.944519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.960865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a627f25c25864076a0e3cb5f77cf755d, user_id=18c75adf08e34786a3bd4a847e4a4c4e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.964248 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.967846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1317/2633] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:56 2022] PUT /identity/v3/projects/cd67f1d0479e4f0893abfc00390db9a0/users/3920e0382d4049359ed208f796d8b9c8/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:56.973610 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.976388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.991036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1317/2634] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:56 2022] PUT /identity/v3/projects/a627f25c25864076a0e3cb5f77cf755d/users/18c75adf08e34786a3bd4a847e4a4c4e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:56.991036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:57.078478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac73ba-1613-4d87-9e3a-361524442fa1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:57.105574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aaed4-c4c0-481c-83ad-cbac8f1727ec admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:57.112368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8963fa-97b7-4133-a4f2-3a0296c6eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:57.112698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8963fa-97b7-4133-a4f2-3a0296c6eb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:57.113074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8963fa-97b7-4133-a4f2-3a0296c6eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:57.113265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8963fa-97b7-4133-a4f2-3a0296c6eb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:57.113935 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8963fa-97b7-4133-a4f2-3a0296c6eb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:57.115752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8963fa-97b7-4133-a4f2-3a0296c6eb99 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:57.138604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1318/2635] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:57.138604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:57.138604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:57.138604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:57.163677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:57.164309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:57.164600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:57.164866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:57.180130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] MFA Rules not processed for user `18c75adf08e34786a3bd4a847e4a4c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:57.181719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:57.181972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:57.243792 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:57.246035 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:57.247706 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:57.258747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1318/2636] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:57.260242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:57.280422 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eaa17-4a0e-49df-97dc-68d2dc31fd33 service glance] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:57.286146 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1319/2637] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:57.293864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:57.294765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:57.295118 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:57.295588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] PATH_INFO: `/v3/projects/cd67f1d0479e4f0893abfc00390db9a0/users/3920e0382d4049359ed208f796d8b9c8/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:57.297965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd67f1d0479e4f0893abfc00390db9a0, user_id=3920e0382d4049359ed208f796d8b9c8, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:57.301551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:57.304372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:57.306760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:57.317311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1319/2638] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:57 2022] PUT /identity/v3/projects/cd67f1d0479e4f0893abfc00390db9a0/users/3920e0382d4049359ed208f796d8b9c8/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:57.527624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3819a-dce0-488f-be53-3d81da3fb342 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:57.673280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3745280-b9c5-480f-907b-de44c106726f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:57.676355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3745280-b9c5-480f-907b-de44c106726f service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:57.676355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3745280-b9c5-480f-907b-de44c106726f service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:57.676878 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3745280-b9c5-480f-907b-de44c106726f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:57.758252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:57.777243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3745280-b9c5-480f-907b-de44c106726f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:57.781045 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3745280-b9c5-480f-907b-de44c106726f service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:57.811421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1320/2639] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:58.204993 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:58.338304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a4350-21c3-4385-9687-3f95a47add81 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:58.343707 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:58.344476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.344783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.345024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] PATH_INFO: `/v3/users/18c75adf08e34786a3bd4a847e4a4c4e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.347197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c75adf08e34786a3bd4a847e4a4c4e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:58.349469 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:58.420332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c75adf08e34786a3bd4a847e4a4c4e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.421408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c75adf08e34786a3bd4a847e4a4c4e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.438889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c75adf08e34786a3bd4a847e4a4c4e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.463869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c75adf08e34786a3bd4a847e4a4c4e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.464959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-17536be1-451c-4170-970d-7c27c0336750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c75adf08e34786a3bd4a847e4a4c4e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.490937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1320/2640] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:58 2022] DELETE /identity/v3/users/18c75adf08e34786a3bd4a847e4a4c4e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:58.498038 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.499036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.500252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.544892 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:58.546135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:58.616913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.616913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.617332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.673514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] MFA Rules not processed for user `3920e0382d4049359ed208f796d8b9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:58.677467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:58.697440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:58.831927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1321/2641] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:58.843437 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:58.844958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:58.847928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1321/2642] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:58.963649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aea586-1d6d-42a8-8841-3bc4ad3649bb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:58.967546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:58.968173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.968481 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.968722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.988004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:58.991096 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.007308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1322/2643] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:59.800579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:59.803104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:59.821872 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82e5b0-a12f-483f-a004-481c0dbd1f12 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:59.834575 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:59.837171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:59.841525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:59.842483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:59.842804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:59.843147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] PATH_INFO: `/v3/projects/a627f25c25864076a0e3cb5f77cf755d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:59.844563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a627f25c25864076a0e3cb5f77cf755d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:59.847724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.994356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a627f25c25864076a0e3cb5f77cf755d', 'request_id': 'req-35614201-a2b8-4e72-9bed-68b1fab61b5b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:00.009678 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a627f25c25864076a0e3cb5f77cf755d', 'request_id': 'req-35614201-a2b8-4e72-9bed-68b1fab61b5b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:00.021347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a627f25c25864076a0e3cb5f77cf755d', 'request_id': 'req-35614201-a2b8-4e72-9bed-68b1fab61b5b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:00.041059 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] The token cache is being invalidate because project a627f25c25864076a0e3cb5f77cf755d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:00.041497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:00.042278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-35614201-a2b8-4e72-9bed-68b1fab61b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:00.044276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1322/2644] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:59 2022] DELETE /identity/v3/projects/a627f25c25864076a0e3cb5f77cf755d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:00.049863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3644-658f-4ed7-a8d8-8c75acef33ff None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.049863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3644-658f-4ed7-a8d8-8c75acef33ff None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.049863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3644-658f-4ed7-a8d8-8c75acef33ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.076370 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff3d6b-e556-4d64-94d3-e68cd484f008 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.078989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a6ea3644-658f-4ed7-a8d8-8c75acef33ff None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:00.081276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a6ea3644-658f-4ed7-a8d8-8c75acef33ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:00.102028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:00.228649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.229508 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.229855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.230372 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.237408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1323/2645] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:00.239007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:00.240171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.240653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.241172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.269089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.272506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:00.273087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.274595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:00.293830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1323/2646] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:00.304140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1324/2647] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:00.311362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.344399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988de3f4-a9df-415c-8a75-38e2f1bcad59 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.363509 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.364732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.365203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.368880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.373730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:00.374931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.378091 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.398861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1324/2648] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:00.405427 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.430784 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc5217-c4a2-4e44-807c-7e7cb97edc6b None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.438459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.440659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.440991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.441646 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.442788 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.446574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.500240 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:00.520174 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1325/2649] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:00 2022] POST /identity/v3/projects => generated 407 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:00.521246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.532175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f00f-b31d-4c57-a7c4-e117946728bb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.676561 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.677318 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.677735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.678227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.678977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.681765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.715311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1325/2650] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:00 2022] POST /identity/v3/users => generated 304 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:00.716463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.737913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ecd94-383f-4311-bd65-bf22d3dd2139 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.745697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.749595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.749595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.749595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.749595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.752081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.774934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1326/2651] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:00.781313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.794394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ded87-a398-424d-bb92-72d24d04c569 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.801297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728dd18-692f-49a0-8789-28216ecd9f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.802123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728dd18-692f-49a0-8789-28216ecd9f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.802583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728dd18-692f-49a0-8789-28216ecd9f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.803193 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728dd18-692f-49a0-8789-28216ecd9f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.803976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728dd18-692f-49a0-8789-28216ecd9f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.806326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728dd18-692f-49a0-8789-28216ecd9f67 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.820275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1326/2652] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:00.825625 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.842504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd01106-a651-4ae9-8304-e18b3f4a9775 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:00.851795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:00.852692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.853175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.853615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] PATH_INFO: `/v3/projects/274c2f823665414d92bfbd622ee95355/users/6502cff29de94e71908f6624e5ad8b3e/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.880525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=274c2f823665414d92bfbd622ee95355, user_id=6502cff29de94e71908f6624e5ad8b3e, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:00.884154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:00.914910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1327/2653] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:00 2022] PUT /identity/v3/projects/274c2f823665414d92bfbd622ee95355/users/6502cff29de94e71908f6624e5ad8b3e/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:00.931574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:00.933931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:00.934544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:00.987691 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:00.989632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:01.002496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] MFA Rules not processed for user `6502cff29de94e71908f6624e5ad8b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:01.005399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:01.090091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831f34-3bdc-4294-b434-7befd7328c2b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.096022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.097457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.097744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.098021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] PATH_INFO: `/v3/users/168f4f7f972548e6b7de4e43073e469c` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.099256 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] RBAC: Authorizing `identity:delete_user(user_id=168f4f7f972548e6b7de4e43073e469c)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.101210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.109806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1327/2654] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:01.116944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.126753 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:01.139132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b545-8891-46da-bc16-68e76f5598c9 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.168925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.169852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.170252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.170701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.171914 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.176136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.193010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168f4f7f972548e6b7de4e43073e469c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.194039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168f4f7f972548e6b7de4e43073e469c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.207865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168f4f7f972548e6b7de4e43073e469c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.220799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1328/2655] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:01 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:01.227021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.231103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168f4f7f972548e6b7de4e43073e469c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.231899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168f4f7f972548e6b7de4e43073e469c'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.248720 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1328/2656] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:00 2022] DELETE /identity/v3/users/168f4f7f972548e6b7de4e43073e469c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:01.311839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3dba7-0f21-42f9-b2fa-065cc99b625b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.421799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.422671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.423089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.423566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.424884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.428306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.467794 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1329/2657] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:01 2022] POST /identity/v3/users => generated 310 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:01.473484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.476135 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:01.497911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad05ede-62bb-48be-a0c9-6709e9b7790e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.514324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.515617 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.516175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.516719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.517740 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.520919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.548995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.552816 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:01.558334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1329/2658] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:01.565844 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.571078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41019fa0-a72d-4f15-8804-5e25f08b4457 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.578265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.578930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.579436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.579951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a510949-6630-419f-b301-a44e5e5b6db5 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.580244 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] PATH_INFO: `/v3/projects/061272abc61045ff95cda9786b9bcf0a` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.581492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=061272abc61045ff95cda9786b9bcf0a)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.583554 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.588857 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.590212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.590595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.590945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] PATH_INFO: `/v3/projects/4800e9d7486448c184be6de9b300a9b8/users/67d8c602f24141de8a41478b8bdbfcba/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.616167 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4800e9d7486448c184be6de9b300a9b8, user_id=67d8c602f24141de8a41478b8bdbfcba, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.619133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.648409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1330/2659] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:01 2022] PUT /identity/v3/projects/4800e9d7486448c184be6de9b300a9b8/users/67d8c602f24141de8a41478b8bdbfcba/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:01.650341 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.662925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:01.671111 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061272abc61045ff95cda9786b9bcf0a', 'request_id': 'req-f3142707-559e-465e-9e82-61e24edb8eea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.682612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061272abc61045ff95cda9786b9bcf0a', 'request_id': 'req-f3142707-559e-465e-9e82-61e24edb8eea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.700142 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061272abc61045ff95cda9786b9bcf0a', 'request_id': 'req-f3142707-559e-465e-9e82-61e24edb8eea'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.711455 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:01.723308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] The token cache is being invalidate because project 061272abc61045ff95cda9786b9bcf0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:01.724434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.726477 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:01.729971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1330/2660] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:01 2022] DELETE /identity/v3/projects/061272abc61045ff95cda9786b9bcf0a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:01.735799 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.739641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:01.772085 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1eeeac-41bc-4bf5-ac88-f9e10e574188 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.881881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3142707-559e-465e-9e82-61e24edb8eea admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:01.889623 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.896846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.897483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.898020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] PATH_INFO: `/v3/users/3920e0382d4049359ed208f796d8b9c8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.911804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3920e0382d4049359ed208f796d8b9c8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.924737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.926964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:01.931876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:01.932444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:01.933356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:01.934046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:01.945453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:01.966798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1331/2661] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:01.975177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:01.978137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:02.009238 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abea38-7061-4820-bccb-2b9fd771cbfc admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:02.018298 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d4617-deb7-48df-a22f-3c87ed1ba65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:02.019097 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d4617-deb7-48df-a22f-3c87ed1ba65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:02.019388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d4617-deb7-48df-a22f-3c87ed1ba65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:02.019664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d4617-deb7-48df-a22f-3c87ed1ba65b admin admin] PATH_INFO: `/v3/projects/4800e9d7486448c184be6de9b300a9b8/users/67d8c602f24141de8a41478b8bdbfcba/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:02.023519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d4617-deb7-48df-a22f-3c87ed1ba65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4800e9d7486448c184be6de9b300a9b8, user_id=67d8c602f24141de8a41478b8bdbfcba, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:02.030510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d4617-deb7-48df-a22f-3c87ed1ba65b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:02.053755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3920e0382d4049359ed208f796d8b9c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.054894 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3920e0382d4049359ed208f796d8b9c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.055756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3920e0382d4049359ed208f796d8b9c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.061409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1332/2662] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:01 2022] PUT /identity/v3/projects/4800e9d7486448c184be6de9b300a9b8/users/67d8c602f24141de8a41478b8bdbfcba/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:02.068683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3920e0382d4049359ed208f796d8b9c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.082119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:02.082569 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:02.083014 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:02.109154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3920e0382d4049359ed208f796d8b9c8'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.156109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1331/2663] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:01 2022] DELETE /identity/v3/users/3920e0382d4049359ed208f796d8b9c8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:02.161666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] MFA Rules not processed for user `67d8c602f24141de8a41478b8bdbfcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:02.164301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:02.344959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1333/2664] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:02 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:02.363042 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:02.365895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:02.462478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ec2bc-e934-4867-a96d-f010e61eb04b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:02.468759 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:02.469404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:02.469752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:02.470082 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:02.492364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:02.493891 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:02.508998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1332/2665] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:02 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:02.593595 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:02.595914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:02.648236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:02.724195 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4ec32-3485-401d-9ea8-89d91b2bd34d None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:02.730543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:02.731418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:02.731815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:02.732036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] PATH_INFO: `/v3/projects/cd67f1d0479e4f0893abfc00390db9a0` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:02.733635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd67f1d0479e4f0893abfc00390db9a0)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:02.737403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:02.783304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:02.785984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:02.801770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b0e44-9077-4ae6-8698-e58f1b818e84 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:02.824017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd67f1d0479e4f0893abfc00390db9a0', 'request_id': 'req-a6722054-a18a-47af-b07b-99a8897642f0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.829408 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:02.838998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd67f1d0479e4f0893abfc00390db9a0', 'request_id': 'req-a6722054-a18a-47af-b07b-99a8897642f0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.852271 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd67f1d0479e4f0893abfc00390db9a0', 'request_id': 'req-a6722054-a18a-47af-b07b-99a8897642f0'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.871157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] The token cache is being invalidate because project cd67f1d0479e4f0893abfc00390db9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:02.871627 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.872508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-a6722054-a18a-47af-b07b-99a8897642f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:02.875109 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1334/2666] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:02 2022] DELETE /identity/v3/projects/cd67f1d0479e4f0893abfc00390db9a0 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:02.883363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbdf42-1cc3-4584-9251-66dfe48697be None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:02.884036 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbdf42-1cc3-4584-9251-66dfe48697be None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:02.884496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbdf42-1cc3-4584-9251-66dfe48697be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:02.895413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:02.896327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:02.896671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:02.897014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:02.899951 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:02.912811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-91bbdf42-1cc3-4584-9251-66dfe48697be None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:02.914067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-91bbdf42-1cc3-4584-9251-66dfe48697be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:03.009169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.010801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1335/2667] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:03.011710 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.021394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.021642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.021845 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.028361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1333/2668] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:02 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:03.050677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:03.051984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:03.060476 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1336/2669] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:03.066824 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.083010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66b109-f3fd-4c3f-9856-77215ef7f97d service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.093145 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.094204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.094712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.095246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.097657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:03.098158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.100970 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.124316 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1334/2670] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:03.127320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.148503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4f0e7-aee5-49c7-950e-15277414a32c None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.156571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.157365 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.157796 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.158178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.160482 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.164266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.219050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1337/2671] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:03 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:03.226113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.244939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aba56-b906-4510-88e6-4d9639d30a74 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.364868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.365740 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.366024 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.366300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.367257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.371037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.419541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1335/2672] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:03 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:03.425755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.445615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd841bf-110d-4825-826a-5d1610623e45 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.454266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.455330 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.455973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.456393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.458609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.459920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.494498 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1338/2673] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:03.501705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.527568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc3850-e03d-4a2f-85f1-6a0b44cd8334 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.535291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92dac-6f43-446d-89f2-5c8579361e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.536123 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92dac-6f43-446d-89f2-5c8579361e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.536531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92dac-6f43-446d-89f2-5c8579361e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.536923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92dac-6f43-446d-89f2-5c8579361e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.537814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb92dac-6f43-446d-89f2-5c8579361e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.540109 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb92dac-6f43-446d-89f2-5c8579361e23 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.559418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1336/2674] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:03.569413 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.585213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42dd7d-4f30-4697-b085-a99be2fb8db1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.595557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.596762 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.597558 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.598484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] PATH_INFO: `/v3/projects/7502a1b893714fcdbabf08c06ce852ea/users/a7ea7fb8daca401a85a88a2fa8288ecd/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.632628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:03.636544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7502a1b893714fcdbabf08c06ce852ea, user_id=a7ea7fb8daca401a85a88a2fa8288ecd, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.639823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.680742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1339/2675] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:03 2022] PUT /identity/v3/projects/7502a1b893714fcdbabf08c06ce852ea/users/a7ea7fb8daca401a85a88a2fa8288ecd/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:03.694880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.695577 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.696294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.737689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] MFA Rules not processed for user `a7ea7fb8daca401a85a88a2fa8288ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:03.739294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:03.777762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:03.823611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1337/2676] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:03.829478 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.908383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff472b-7c60-4583-83a7-1c50dd509a81 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:03.923235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:03.923891 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:03.924207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:03.924521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:03.925345 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:03.927352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:03.959261 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1340/2677] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:03 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:03.963696 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:03.980806 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662792e-c9b0-4040-96df-c4b7c236c7ef None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:04.084649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:04.085612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:04.085612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:04.087019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:04.087019 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:04.088913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:04.123514 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1338/2678] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:03 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:04.129169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:04.150892 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db96c5-520d-46ce-906f-fe231ae0b909 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:04.161033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:04.163215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:04.163717 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:04.164203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:04.165391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:04.168518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:04.196426 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1341/2679] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:04.198076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:04.211101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330804a4-bf46-424d-bfca-2a08e1e54af6 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:04.216548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:04.217219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:04.217501 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:04.217771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] PATH_INFO: `/v3/projects/191330a51b1b446aa0fef1384b1892c4/users/5924cc72f6d2447a97440191418ea732/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:04.239088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] RBAC: Authorizing `identity:create_grant(project_id=191330a51b1b446aa0fef1384b1892c4, user_id=5924cc72f6d2447a97440191418ea732, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:04.241836 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:04.264819 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1339/2680] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:04 2022] PUT /identity/v3/projects/191330a51b1b446aa0fef1384b1892c4/users/5924cc72f6d2447a97440191418ea732/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:04.271375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:04.286794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3bbd9-72ca-4223-91a4-45c1ba256226 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:04.484833 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71623de6-0f7e-4710-9755-b0c85121437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:04.485630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623de6-0f7e-4710-9755-b0c85121437a admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:04.486010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623de6-0f7e-4710-9755-b0c85121437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:04.486437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71623de6-0f7e-4710-9755-b0c85121437a admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:04.491556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623de6-0f7e-4710-9755-b0c85121437a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:04.496140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71623de6-0f7e-4710-9755-b0c85121437a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:04.515551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1342/2681] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:04.520983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:04.539436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7e6cd-8abe-4061-ae55-3e8d89a4097e admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:04.546236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:04.547198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:04.547678 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:04.548166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] PATH_INFO: `/v3/projects/191330a51b1b446aa0fef1384b1892c4/users/5924cc72f6d2447a97440191418ea732/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:04.551465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=191330a51b1b446aa0fef1384b1892c4, user_id=5924cc72f6d2447a97440191418ea732, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:04.554176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:04.566258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1340/2682] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:04 2022] PUT /identity/v3/projects/191330a51b1b446aa0fef1384b1892c4/users/5924cc72f6d2447a97440191418ea732/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:04.585983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:04.586485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:04.587488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:04.632896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] MFA Rules not processed for user `5924cc72f6d2447a97440191418ea732`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:04.634131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:04.717418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:04.779020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1343/2683] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:04.831620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:04.834410 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:05.002714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ea869-4a45-4cb8-883b-2cf82b2df1cf admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:05.009233 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b58223-c497-4b33-980a-2469afefc20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:05.009863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b58223-c497-4b33-980a-2469afefc20f service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:05.010174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b58223-c497-4b33-980a-2469afefc20f service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:05.010452 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b58223-c497-4b33-980a-2469afefc20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:05.023713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b58223-c497-4b33-980a-2469afefc20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:05.025279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b58223-c497-4b33-980a-2469afefc20f service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:05.039081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1341/2684] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:05.633515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:05.774456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e38dda-5592-48e6-97c6-f469f33cede9 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:05.783128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:05.785769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:05.786454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:05.787123 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] PATH_INFO: `/v3/users/a7ea7fb8daca401a85a88a2fa8288ecd` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:05.789145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ea7fb8daca401a85a88a2fa8288ecd)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:05.792119 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:05.884631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ea7fb8daca401a85a88a2fa8288ecd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:05.889333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ea7fb8daca401a85a88a2fa8288ecd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:05.918526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ea7fb8daca401a85a88a2fa8288ecd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:05.940313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ea7fb8daca401a85a88a2fa8288ecd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:05.941104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ea7fb8daca401a85a88a2fa8288ecd'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:05.971871 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1344/2685] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:05 2022] DELETE /identity/v3/users/a7ea7fb8daca401a85a88a2fa8288ecd => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:05.978005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:05.978781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:05.978781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:06.009719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:06.011694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:06.041270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:06.112813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1342/2686] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:06.134169 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:06.135836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:06.227504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca056-9a02-46f2-a5d2-60fb435686ef admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:06.232348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:06.232983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:06.233308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:06.233524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:06.250711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:06.252201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:06.263693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1345/2687] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:06.770596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:06.772190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:06.786209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c47bc-01e3-4a3d-a6ab-7afcb547cf9d None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:06.791390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:06.792790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:06.793072 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:06.793483 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] PATH_INFO: `/v3/projects/7502a1b893714fcdbabf08c06ce852ea` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:06.794708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7502a1b893714fcdbabf08c06ce852ea)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:06.796844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:06.887668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7502a1b893714fcdbabf08c06ce852ea', 'request_id': 'req-c3c36346-ac6e-4163-a397-e5cb043d205a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:06.896022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7502a1b893714fcdbabf08c06ce852ea', 'request_id': 'req-c3c36346-ac6e-4163-a397-e5cb043d205a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:06.905498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7502a1b893714fcdbabf08c06ce852ea', 'request_id': 'req-c3c36346-ac6e-4163-a397-e5cb043d205a'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:06.920671 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] The token cache is being invalidate because project 7502a1b893714fcdbabf08c06ce852ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:06.921001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:06.921861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:06.923738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1343/2688] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:06 2022] DELETE /identity/v3/projects/7502a1b893714fcdbabf08c06ce852ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:06.928549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:06.943692 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a709a35-9ef8-48e3-9002-160550bb6b51 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:06.952559 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:07.035404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:07.036234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:07.036671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:07.038230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] PATH_INFO: `/v3/users/5924cc72f6d2447a97440191418ea732` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:07.045354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5924cc72f6d2447a97440191418ea732)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:07.045354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:07.140816 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5924cc72f6d2447a97440191418ea732'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.141582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5924cc72f6d2447a97440191418ea732'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.155208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5924cc72f6d2447a97440191418ea732'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.174543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5924cc72f6d2447a97440191418ea732'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.175489 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-55d66839-0a8e-49c8-98a9-184544d5e8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5924cc72f6d2447a97440191418ea732'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.195809 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1346/2689] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:06 2022] DELETE /identity/v3/users/5924cc72f6d2447a97440191418ea732 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:07.631263 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:07.634207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:07.649461 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36346-ac6e-4163-a397-e5cb043d205a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:07.743177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:07.743823 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:07.744101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:07.744381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] PATH_INFO: `/v3/projects/191330a51b1b446aa0fef1384b1892c4` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:07.745846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191330a51b1b446aa0fef1384b1892c4)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:07.748609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:07.840620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191330a51b1b446aa0fef1384b1892c4', 'request_id': 'req-93b37849-4242-493a-ba51-a93e1869d099'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.843171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:07.848883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191330a51b1b446aa0fef1384b1892c4', 'request_id': 'req-93b37849-4242-493a-ba51-a93e1869d099'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.861296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191330a51b1b446aa0fef1384b1892c4', 'request_id': 'req-93b37849-4242-493a-ba51-a93e1869d099'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.880770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] The token cache is being invalidate because project 191330a51b1b446aa0fef1384b1892c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:07.881296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.882600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-93b37849-4242-493a-ba51-a93e1869d099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:07.890934 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1344/2690] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:07 2022] DELETE /identity/v3/projects/191330a51b1b446aa0fef1384b1892c4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:07.893631 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:07.894125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:07.894604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:07.925171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:07.926873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:08.056087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1347/2691] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:08.065631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.065967 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.066121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.089978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:08.091359 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:08.111943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1345/2692] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:08.113561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.128152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20459e8a-4f0c-404b-b336-93d93314ac45 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.138367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.139672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.140061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.141145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.142105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:08.142727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.146922 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.178297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1348/2693] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:08.182182 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.199695 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db38fd-6c4f-4e99-9393-5f174055e79e None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.208553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.209528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.210064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.210568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.211729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.215580 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.262162 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1346/2694] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:08 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:08.267942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.285636 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c10fd-7260-41b8-925c-53ba76edb8fa admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.302017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:08.368439 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.369249 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.369665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.370227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.371102 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.373962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.411590 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1349/2695] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:08 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:08.416636 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.431278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777e704-3d25-4521-bcee-8197851c51fc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.440899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.442555 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.443159 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.443496 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.444670 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.447781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.462641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1347/2696] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:08.467434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.479486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33b592-09a3-4231-87f1-988158d9ecd0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.485839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66e67d-ff1d-4003-9fa8-b96061c3c97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.486463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e67d-ff1d-4003-9fa8-b96061c3c97f admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.487323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e67d-ff1d-4003-9fa8-b96061c3c97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.487594 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66e67d-ff1d-4003-9fa8-b96061c3c97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.488358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66e67d-ff1d-4003-9fa8-b96061c3c97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.490126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66e67d-ff1d-4003-9fa8-b96061c3c97f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.508826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1350/2697] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:08.511435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.523236 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e521c3-235a-4e97-bf03-d56ea506223b admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.528493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.529156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.529436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.529714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] PATH_INFO: `/v3/projects/2024508ff27942308bfc1f4f6f9ef9d9/users/83411a8a7eee4d889469b1cf7c4977b1/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.557278 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2024508ff27942308bfc1f4f6f9ef9d9, user_id=83411a8a7eee4d889469b1cf7c4977b1, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.560583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.582163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1348/2698] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:08 2022] PUT /identity/v3/projects/2024508ff27942308bfc1f4f6f9ef9d9/users/83411a8a7eee4d889469b1cf7c4977b1/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:08.600942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.601375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.601825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.644039 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] MFA Rules not processed for user `83411a8a7eee4d889469b1cf7c4977b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:08.645649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:08.744732 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1351/2699] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:08.749694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.839478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b99b1b-f535-493a-976e-45385a94f538 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:08.844723 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:08.845339 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:08.845615 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:08.845890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:08.846565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:08.848520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:08.885535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1349/2700] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:08 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:08.889840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:08.924231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363e85-c823-4b6e-a55d-4cee270587aa None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:09.032674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:09.034208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.034496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.034772 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.036258 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:09.038247 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:09.068699 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1352/2701] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:08 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:09.072984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:09.085435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815040-432d-4453-8b4a-08b0f20967f9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:09.090282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:09.090938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.091229 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.091499 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.092196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:09.093886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:09.110084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1350/2702] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:09.115300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:09.132565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48709-f142-4f28-9741-6cc39cd0c2c8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:09.144408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:09.145262 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.145689 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.146145 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] PATH_INFO: `/v3/projects/45d89449ad904795807f879cce874c30/users/8494458b8bf84011b3fa4719bde5a685/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.170666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:09.181203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d89449ad904795807f879cce874c30, user_id=8494458b8bf84011b3fa4719bde5a685, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:09.186867 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:09.225416 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1353/2703] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:09 2022] PUT /identity/v3/projects/45d89449ad904795807f879cce874c30/users/8494458b8bf84011b3fa4719bde5a685/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:09.227546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:09.252256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d396376-ac74-4e23-9c55-588e8307e979 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:09.332829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:09.366131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8270da-a1be-4b75-87fd-c9d5f2332405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:09.367337 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8270da-a1be-4b75-87fd-c9d5f2332405 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.367946 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8270da-a1be-4b75-87fd-c9d5f2332405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.368348 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8270da-a1be-4b75-87fd-c9d5f2332405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.369376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8270da-a1be-4b75-87fd-c9d5f2332405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:09.372465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8270da-a1be-4b75-87fd-c9d5f2332405 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:09.403064 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1351/2704] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:09.407677 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:09.430157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09cd9a-2f0f-41ad-acf9-84c07bcb0543 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:09.440433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:09.441840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.442310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.443253 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] PATH_INFO: `/v3/projects/45d89449ad904795807f879cce874c30/users/8494458b8bf84011b3fa4719bde5a685/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.447462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d89449ad904795807f879cce874c30, user_id=8494458b8bf84011b3fa4719bde5a685, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:09.453747 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:09.493970 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1354/2705] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:09 2022] PUT /identity/v3/projects/45d89449ad904795807f879cce874c30/users/8494458b8bf84011b3fa4719bde5a685/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:09.505169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.505845 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.506211 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.598767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] MFA Rules not processed for user `8494458b8bf84011b3fa4719bde5a685`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:09.601205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:09.735136 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1352/2706] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:09.755492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:09.757604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:09.844785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d01f2-c5bb-4a5e-ad01-56ad8db3969d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:09.849061 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:09.849793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:09.850096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:09.850381 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:09.873190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:09.874665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:09.895989 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1355/2707] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:09.983511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:10.082688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a837c-94c8-4cc1-9797-e31805639f99 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:10.088394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:10.089040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:10.089330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:10.089754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] PATH_INFO: `/v3/users/83411a8a7eee4d889469b1cf7c4977b1` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:10.091022 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83411a8a7eee4d889469b1cf7c4977b1)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:10.092972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:10.157264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83411a8a7eee4d889469b1cf7c4977b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:10.158207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83411a8a7eee4d889469b1cf7c4977b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:10.160822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83411a8a7eee4d889469b1cf7c4977b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:10.174448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83411a8a7eee4d889469b1cf7c4977b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:10.199849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9481ae86-98d5-4d61-8dc7-b80dcc643785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83411a8a7eee4d889469b1cf7c4977b1'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:10.203837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:10.223446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1353/2708] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:09 2022] DELETE /identity/v3/users/83411a8a7eee4d889469b1cf7c4977b1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:10.228152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:10.229300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:10.229606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:10.229909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:10.230278 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:10.254766 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:10.256171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:10.335939 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe9f9f-652d-4f05-b071-7ce0719a478d service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:10.350938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1354/2709] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:10.363961 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:10.366276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:10.427986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:10.429027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:10.429556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:10.430041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:10.432904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:10.442297 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682e64-c736-4b51-9388-405a3f521d17 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:10.445406 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:10.447104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:10.448528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:10.449414 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:10.449947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:10.450394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:10.462434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1356/2710] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:10.468178 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:10.470625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:10.488197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1355/2711] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:10.857150 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:10.859198 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:10.894750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ccc7e-ec00-4ec0-8fff-921e58330569 service placement] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:10.903008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:10.904614 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:10.905430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:10.905980 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] PATH_INFO: `/v3/projects/2024508ff27942308bfc1f4f6f9ef9d9` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:10.908317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2024508ff27942308bfc1f4f6f9ef9d9)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:10.912009 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:10.920171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:10.991819 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2024508ff27942308bfc1f4f6f9ef9d9', 'request_id': 'req-2f5641ae-0f37-457c-a75f-38582e63ec55'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.001359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2024508ff27942308bfc1f4f6f9ef9d9', 'request_id': 'req-2f5641ae-0f37-457c-a75f-38582e63ec55'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.010472 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2024508ff27942308bfc1f4f6f9ef9d9', 'request_id': 'req-2f5641ae-0f37-457c-a75f-38582e63ec55'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.027420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] The token cache is being invalidate because project 2024508ff27942308bfc1f4f6f9ef9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:11.027759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.028481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.030013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1357/2712] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:10 2022] DELETE /identity/v3/projects/2024508ff27942308bfc1f4f6f9ef9d9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:11.040708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:11.053683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02064cf2-4d08-4706-b628-1f310e4a2c4f service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:11.148524 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:11.149594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:11.150215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:11.150587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] PATH_INFO: `/v3/users/8494458b8bf84011b3fa4719bde5a685` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:11.152004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8494458b8bf84011b3fa4719bde5a685)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:11.154360 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:11.217401 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:INSERT=1,keystone:DELETE=6,keystone:SELECT=93 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:11.232583 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8494458b8bf84011b3fa4719bde5a685'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.233491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8494458b8bf84011b3fa4719bde5a685'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.234288 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8494458b8bf84011b3fa4719bde5a685'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.251083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8494458b8bf84011b3fa4719bde5a685'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.279516 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-9fcb5fbe-28e9-4ada-aa42-f8620eff0666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8494458b8bf84011b3fa4719bde5a685'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.295868 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1356/2713] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:11 2022] DELETE /identity/v3/users/8494458b8bf84011b3fa4719bde5a685 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:11.673684 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:11.676305 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:11.698125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5641ae-0f37-457c-a75f-38582e63ec55 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:11.775193 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:11.775768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:11.776046 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:11.776349 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] PATH_INFO: `/v3/projects/45d89449ad904795807f879cce874c30` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:11.777550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d89449ad904795807f879cce874c30)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:11.779661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:11.841990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d89449ad904795807f879cce874c30', 'request_id': 'req-6903a056-290f-4d58-b8be-5fc2e841f559'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.848525 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d89449ad904795807f879cce874c30', 'request_id': 'req-6903a056-290f-4d58-b8be-5fc2e841f559'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.855110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d89449ad904795807f879cce874c30', 'request_id': 'req-6903a056-290f-4d58-b8be-5fc2e841f559'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.871432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] The token cache is being invalidate because project 45d89449ad904795807f879cce874c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:11.871786 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.872515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6903a056-290f-4d58-b8be-5fc2e841f559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:11.874530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1358/2714] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:11 2022] DELETE /identity/v3/projects/45d89449ad904795807f879cce874c30 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:11.880369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:11.880658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:11.880932 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:11.903540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:11.904788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:11.995852 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1357/2715] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:12.008181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.008459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.008728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.030437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:12.031637 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:12.040775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1359/2716] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:12.046174 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.063256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b4cb55-6837-4a0f-af80-11c67a309f62 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.071283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.072217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.072681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.072985 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.073822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:12.074177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.077553 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.102048 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1358/2717] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:12.106899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.121114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfdb90-6a82-44c4-bb48-920ce07a4724 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.129212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.130178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.130642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.131140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.132237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.135701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.162133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:12.181849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1360/2718] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:12.186235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.198448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189821-8ebe-481b-a426-fcbcfa2ae967 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.270929 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.271600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.271880 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.272150 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.272821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.275102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.310330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1359/2719] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:12.315333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.334479 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281645c-f5e2-4b66-8af9-dcf40d8e47f1 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.341820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.342571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.343022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.343454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.344269 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.346801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.358719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1361/2720] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:12.363028 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.374959 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad3cad-6013-44ae-b9d9-6f4bec5e5c09 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.383557 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d50783-8dce-400c-acae-74ed9f293035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.384604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50783-8dce-400c-acae-74ed9f293035 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.385113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50783-8dce-400c-acae-74ed9f293035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.385621 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50783-8dce-400c-acae-74ed9f293035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.386796 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d50783-8dce-400c-acae-74ed9f293035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.389954 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d50783-8dce-400c-acae-74ed9f293035 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.405308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1360/2721] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:12.411162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.427217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23764ed-5041-4c0a-8ba0-4970db2e6168 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.441904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.441904 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.442680 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.442964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] PATH_INFO: `/v3/projects/185777b39d5a4e5c8fd917930996c6a2/users/9dfc245614a5459593fd2dc280ca8fc6/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.471194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185777b39d5a4e5c8fd917930996c6a2, user_id=9dfc245614a5459593fd2dc280ca8fc6, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.475207 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.492178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1362/2722] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:12 2022] PUT /identity/v3/projects/185777b39d5a4e5c8fd917930996c6a2/users/9dfc245614a5459593fd2dc280ca8fc6/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:12.505556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.505556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.505556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.539080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.540863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:12.549748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] MFA Rules not processed for user `9dfc245614a5459593fd2dc280ca8fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:12.551134 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:12.561976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:12.641480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1361/2723] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:12.645533 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa340-7817-4a60-82df-1917c77165b7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.646169 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.650827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.651755 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.652311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.652745 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] PATH_INFO: `/v3/users/6502cff29de94e71908f6624e5ad8b3e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.654639 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6502cff29de94e71908f6624e5ad8b3e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.657041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.659436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ae73c-b60d-463d-8d9b-61faf53fe0cb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.670969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.671911 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.672209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.672648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.675724 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.677680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.718805 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1362/2724] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:12.726341 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.746519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6502cff29de94e71908f6624e5ad8b3e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:12.747545 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6502cff29de94e71908f6624e5ad8b3e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:12.759628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6502cff29de94e71908f6624e5ad8b3e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:12.777141 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6502cff29de94e71908f6624e5ad8b3e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:12.778037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-51528e7b-a937-4f67-99cf-e3475c21af5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6502cff29de94e71908f6624e5ad8b3e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:12.800423 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1363/2725] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:12 2022] DELETE /identity/v3/users/6502cff29de94e71908f6624e5ad8b3e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:12.805255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.805530 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.805811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.823991 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667926a-45da-4a59-9bfb-c19a9d5dd135 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.834861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:12.836705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:12.888547 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:12.910926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.911546 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.911822 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.912089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.912729 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.914663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:12.925783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1364/2726] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:12.938598 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.940956 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:12.952282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1363/2727] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:12 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:12.954083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:12.957376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:12.974957 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a778d33-d8e7-4701-8232-17f2d790bfbb admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:12.982312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:12.983218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:12.983218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:12.983218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:12.983218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:12.988049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.000532 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1364/2728] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:13.005478 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.016254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2ed94-b316-4f93-a05f-b909dc0c81e8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.024195 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.025209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.025493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.025780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] PATH_INFO: `/v3/projects/8a35a426056a4d5f9d762f692d7e510d/users/1f8c683b81f340fd8f90a1134c82a200/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.046492 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:13.055705 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a35a426056a4d5f9d762f692d7e510d, user_id=1f8c683b81f340fd8f90a1134c82a200, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.061755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.071761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9f95-5c31-4c5c-8ff6-22cc2a294963 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.076537 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.077506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.077896 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.078303 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.082120 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1365/2729] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:13 2022] PUT /identity/v3/projects/8a35a426056a4d5f9d762f692d7e510d/users/1f8c683b81f340fd8f90a1134c82a200/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:13.086442 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.091838 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.093502 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.104801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1365/2730] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:13.176869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f93cf0-cf98-49bd-b77d-ae0b7d954997 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.183791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed920f01-e8c2-4cc9-9a4c-dac043facd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.184534 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed920f01-e8c2-4cc9-9a4c-dac043facd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.184813 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed920f01-e8c2-4cc9-9a4c-dac043facd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.185026 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed920f01-e8c2-4cc9-9a4c-dac043facd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.185731 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed920f01-e8c2-4cc9-9a4c-dac043facd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.187846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed920f01-e8c2-4cc9-9a4c-dac043facd69 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.210462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1366/2731] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:13.221936 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.239497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f6459-8e33-4f78-ae7c-ae81a720cbac service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.247967 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.248804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.249311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.250031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] PATH_INFO: `/v3/projects/8a35a426056a4d5f9d762f692d7e510d/users/1f8c683b81f340fd8f90a1134c82a200/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.254006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a35a426056a4d5f9d762f692d7e510d, user_id=1f8c683b81f340fd8f90a1134c82a200, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.257624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.275088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1366/2732] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:13 2022] PUT /identity/v3/projects/8a35a426056a4d5f9d762f692d7e510d/users/1f8c683b81f340fd8f90a1134c82a200/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:13.293256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.293754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.294089 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.339335 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] MFA Rules not processed for user `1f8c683b81f340fd8f90a1134c82a200`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:13.341298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:13.473330 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1367/2733] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:13.487229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.489224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:13.493484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.496205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:13.618619 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d61b9-8482-49a0-9063-85062ca34fd6 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.621458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a560307-84bc-4b45-9ed6-3b8305660c53 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.624640 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.625722 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.626220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.627010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.630058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.631214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.631728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.632435 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] PATH_INFO: `/v3/projects/274c2f823665414d92bfbd622ee95355` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.634738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=274c2f823665414d92bfbd622ee95355)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.638345 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.645248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.648313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.659690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:13.666588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1367/2734] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:13.754055 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274c2f823665414d92bfbd622ee95355', 'request_id': 'req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:13.765406 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274c2f823665414d92bfbd622ee95355', 'request_id': 'req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:13.778139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274c2f823665414d92bfbd622ee95355', 'request_id': 'req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:13.797733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.804181 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] The token cache is being invalidate because project 274c2f823665414d92bfbd622ee95355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:13.804680 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:13.805896 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:13.808732 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1368/2735] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:13 2022] DELETE /identity/v3/projects/274c2f823665414d92bfbd622ee95355 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:13.813166 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:13.813862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fd736-40e7-4eb6-96f7-dc089fa9d4b6 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.816450 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:13.891430 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.894328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.894328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.894328 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.896122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:13.903418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:13.958493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eddf7-81b2-4c48-afe8-28b2a5f63605 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:13.966685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:13.967793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:13.968255 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:13.968733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] PATH_INFO: `/v3/users/67d8c602f24141de8a41478b8bdbfcba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:13.970983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d8c602f24141de8a41478b8bdbfcba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.974077 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:13.997644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:13.999451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:14.014437 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1368/2736] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:14.059882 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d8c602f24141de8a41478b8bdbfcba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.061764 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d8c602f24141de8a41478b8bdbfcba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.062686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d8c602f24141de8a41478b8bdbfcba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.077993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d8c602f24141de8a41478b8bdbfcba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.097090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d8c602f24141de8a41478b8bdbfcba'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.117327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1369/2737] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:13 2022] DELETE /identity/v3/users/67d8c602f24141de8a41478b8bdbfcba => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:14.461765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:14.464137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:14.489011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235d726-d088-4d61-956e-47155ce7e604 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:14.543399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:14.545613 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:14.589405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:14.614561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:14.620518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:14.620518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:14.620518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] PATH_INFO: `/v3/users/9dfc245614a5459593fd2dc280ca8fc6` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:14.620518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfc245614a5459593fd2dc280ca8fc6)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:14.624682 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:14.683653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e118b7-8596-4618-b3ce-6dc086ff6d64 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:14.693081 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:14.694290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:14.694741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:14.695291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] PATH_INFO: `/v3/projects/4800e9d7486448c184be6de9b300a9b8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:14.697351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4800e9d7486448c184be6de9b300a9b8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:14.700489 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:14.703520 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfc245614a5459593fd2dc280ca8fc6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.706768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfc245614a5459593fd2dc280ca8fc6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.720080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfc245614a5459593fd2dc280ca8fc6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.740671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfc245614a5459593fd2dc280ca8fc6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.742065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f46b6465-11c3-4479-a789-b4f4efa88a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfc245614a5459593fd2dc280ca8fc6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.761998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1369/2738] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:14 2022] DELETE /identity/v3/users/9dfc245614a5459593fd2dc280ca8fc6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:14.766937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:14.767310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:14.767497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:14.789005 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4800e9d7486448c184be6de9b300a9b8', 'request_id': 'req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.800312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:14.801648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:14.804192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4800e9d7486448c184be6de9b300a9b8', 'request_id': 'req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.818206 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4800e9d7486448c184be6de9b300a9b8', 'request_id': 'req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.835293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:14.836697 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] The token cache is being invalidate because project 4800e9d7486448c184be6de9b300a9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:14.837170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.838175 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e9ee80b3-d2d9-4407-9c8e-e12d11a01962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:14.840688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1370/2739] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:14 2022] DELETE /identity/v3/projects/4800e9d7486448c184be6de9b300a9b8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:14.848069 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69cc5-e01a-4ebd-b7e6-6493b0289427 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:14.848493 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69cc5-e01a-4ebd-b7e6-6493b0289427 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:14.848949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69cc5-e01a-4ebd-b7e6-6493b0289427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:14.885944 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-d7b69cc5-e01a-4ebd-b7e6-6493b0289427 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:14.887898 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d7b69cc5-e01a-4ebd-b7e6-6493b0289427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:14.913503 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1370/2740] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:14.927701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:14.929981 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:15.002563 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1371/2741] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:15.016034 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.016458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.016770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.040518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:15.042270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:15.051218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1372/2742] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:15.057498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.062139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69880ff4-e6ed-4759-8beb-d5b4c1d2a17e None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.068348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.068895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ed2b6-e3e3-4c9c-bbeb-b6503ce325cd None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.069635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.069635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.069635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.076527 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.077573 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.078087 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.078591 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.079908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:15.081067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.083089 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.084531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.086393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.107660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1371/2743] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:15.112227 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1373/2744] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:15.120508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.139131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d3a49-009d-41fb-a452-b5af66c436b7 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.157730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.157730 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.158768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.158768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.159402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.163181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.213192 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1372/2745] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:15 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:15.219258 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.236296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8747f-1a2b-4636-804b-2ce9892b7447 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.296523 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.348901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.350061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.350061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.350352 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.350851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.353284 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.394156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1374/2746] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:15 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:15.401313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.428724 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca0ff0-5eef-4ea8-aff3-f46ea2a525de admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.438117 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.439254 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.439794 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.440322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.441940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.446185 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.470604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1373/2747] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:15.473203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.488265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b4319-6c6a-43f0-b159-e074bde11e39 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.494456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.495712 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.496144 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.496551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.497479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.500075 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.545090 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.547515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1375/2748] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:15.560876 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c87a0-c0ce-4fde-ba6a-a9df5eb6d8d7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.568166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b668d-c5a0-4e72-bc95-3d6dac5cc032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.569347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b668d-c5a0-4e72-bc95-3d6dac5cc032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.569882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b668d-c5a0-4e72-bc95-3d6dac5cc032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.570320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b668d-c5a0-4e72-bc95-3d6dac5cc032 admin admin] PATH_INFO: `/v3/projects/e2ee05fa6cb149f5a1e7a78d8a29249c/users/3cb3c7ec478b48e1924fe890bf9d26b5/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.607276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b668d-c5a0-4e72-bc95-3d6dac5cc032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ee05fa6cb149f5a1e7a78d8a29249c, user_id=3cb3c7ec478b48e1924fe890bf9d26b5, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.607276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b668d-c5a0-4e72-bc95-3d6dac5cc032 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.625164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.629831 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:15.646439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6351cfd-7e56-4e91-b33a-7d49208f3fac admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.654869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1374/2749] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:15 2022] PUT /identity/v3/projects/e2ee05fa6cb149f5a1e7a78d8a29249c/users/3cb3c7ec478b48e1924fe890bf9d26b5/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:15.673902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.675697 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.676297 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.728501 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] MFA Rules not processed for user `3cb3c7ec478b48e1924fe890bf9d26b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:15.730499 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:15.761544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.763989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.763989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.763989 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] PATH_INFO: `/v3/projects/185777b39d5a4e5c8fd917930996c6a2` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.766625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] RBAC: Authorizing `identity:delete_project(project_id=185777b39d5a4e5c8fd917930996c6a2)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.770993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:15.849686 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1375/2750] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:15.851515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185777b39d5a4e5c8fd917930996c6a2', 'request_id': 'req-d95c9649-f8ce-402d-88bf-e451ea42392f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:15.855761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.863216 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185777b39d5a4e5c8fd917930996c6a2', 'request_id': 'req-d95c9649-f8ce-402d-88bf-e451ea42392f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:15.864918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.874624 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185777b39d5a4e5c8fd917930996c6a2', 'request_id': 'req-d95c9649-f8ce-402d-88bf-e451ea42392f'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:15.875375 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828326-7f41-44c6-ba8b-8fc8c21e6bc5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:15.892597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] The token cache is being invalidate because project 185777b39d5a4e5c8fd917930996c6a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:15.894074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:15.895050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:15.897404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1376/2751] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:15 2022] DELETE /identity/v3/projects/185777b39d5a4e5c8fd917930996c6a2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:15.903088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:15.905128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:15.981688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.984839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:15.985875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:15.986459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:15.988683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:15.989888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:15.992323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.024998 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c9649-f8ce-402d-88bf-e451ea42392f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.027591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1376/2752] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:15 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:16.032456 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.035736 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:16.115539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.116427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.116771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.117308 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] PATH_INFO: `/v3/users/1f8c683b81f340fd8f90a1134c82a200` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.118609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f8c683b81f340fd8f90a1134c82a200)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.120685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.135683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe4906-e205-46ac-b35c-fc610b80dc36 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.145197 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.146320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.146804 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.147335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.148529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.152373 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.184424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1377/2753] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:16 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:16.190321 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.208660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e026ae9-c818-41da-b868-bf372a2c2de7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.213268 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f8c683b81f340fd8f90a1134c82a200'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.214267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f8c683b81f340fd8f90a1134c82a200'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.215084 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f8c683b81f340fd8f90a1134c82a200'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.215283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.215962 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.216388 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.216769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.217623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.219738 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.225570 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f8c683b81f340fd8f90a1134c82a200'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.234907 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1378/2754] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:16.238310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.244074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:16.244863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f8c683b81f340fd8f90a1134c82a200'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.260227 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cf6fc-a79e-42a4-8412-35059da14a20 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.262112 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1377/2755] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:15 2022] DELETE /identity/v3/users/1f8c683b81f340fd8f90a1134c82a200 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:16.345340 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c050c-cb85-41a5-ba0b-7fb9eccdce86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.347768 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c050c-cb85-41a5-ba0b-7fb9eccdce86 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.348248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c050c-cb85-41a5-ba0b-7fb9eccdce86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.348527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c050c-cb85-41a5-ba0b-7fb9eccdce86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.349255 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c050c-cb85-41a5-ba0b-7fb9eccdce86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.352053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c050c-cb85-41a5-ba0b-7fb9eccdce86 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.364969 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1379/2756] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:16.370124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.389686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dce4-ed4a-49e3-a90c-03a28a28a40a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.397751 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.398947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.399427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.399895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] PATH_INFO: `/v3/projects/09c560ce4aa24d05b2c807ac38e0d7d3/users/2b4b3b008d7d404eb4f8e8bf09688c90/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.419378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c560ce4aa24d05b2c807ac38e0d7d3, user_id=2b4b3b008d7d404eb4f8e8bf09688c90, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.424947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.443984 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1378/2757] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:16 2022] PUT /identity/v3/projects/09c560ce4aa24d05b2c807ac38e0d7d3/users/2b4b3b008d7d404eb4f8e8bf09688c90/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:16.460596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.461032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.461454 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.471638 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.508898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] MFA Rules not processed for user `2b4b3b008d7d404eb4f8e8bf09688c90`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:16.510653 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:16.616512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1380/2758] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:16.622073 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.692050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.693136 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:16.722673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.788397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0ead7-24a2-4791-a89a-3408fdad3a30 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.798047 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.799119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.799494 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.799918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.802905 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.807617 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.853488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71601caa-948d-4897-8f6d-35ca3ccbd375 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.862349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1379/2759] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:16 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:16.863770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:16.864014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:16.864513 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:16.865058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:16.865588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] PATH_INFO: `/v3/projects/8a35a426056a4d5f9d762f692d7e510d` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:16.867535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a35a426056a4d5f9d762f692d7e510d)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:16.870776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:16.956436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a35a426056a4d5f9d762f692d7e510d', 'request_id': 'req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.973251 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f0f0d-b8e7-4c11-95d0-c1c62dc73f36 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:16.977410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a35a426056a4d5f9d762f692d7e510d', 'request_id': 'req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:16.993991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a35a426056a4d5f9d762f692d7e510d', 'request_id': 'req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:17.027308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] The token cache is being invalidate because project 8a35a426056a4d5f9d762f692d7e510d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:17.027925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:17.029092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5d6c10f5-8480-44e6-aeec-fd3998ea74cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:17.036029 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1381/2760] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:16 2022] DELETE /identity/v3/projects/8a35a426056a4d5f9d762f692d7e510d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:17.044657 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf122a8-f36c-41de-901d-7ddf2d7fc0ed None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.045242 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf122a8-f36c-41de-901d-7ddf2d7fc0ed None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.045624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf122a8-f36c-41de-901d-7ddf2d7fc0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.086497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9bf122a8-f36c-41de-901d-7ddf2d7fc0ed None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:17.088060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9bf122a8-f36c-41de-901d-7ddf2d7fc0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:17.094350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.096272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.096728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.097290 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.098257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.100027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1382/2761] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:17.102381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.116068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.117122 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.117551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.141427 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1380/2762] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:16 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:17.148092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:17.148956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.150204 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:17.153431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:17.163791 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1383/2763] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:17.168364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.178322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8565159-1950-46c5-944f-65d4461b2535 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.183822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22288da0-30b4-42df-ad16-85f6db83f09a None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.187983 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.189119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.189684 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.191666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.192707 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.192887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.193694 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.194111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.194539 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.195701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:17.196212 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.196404 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.198982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.203248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.220036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1381/2764] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:17.222520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.223588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1384/2765] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:07:17.228306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.245020 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84894271-9ac8-44b4-9d74-204e9db85a3f admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.245521 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac423a-9e60-4995-a2ee-81c031ac3327 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.251529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.252393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.252832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.253158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.254041 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.255503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.256367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.256655 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.257153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.257632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] PATH_INFO: `/v3/projects/542e8ff48e7c425eb5c1cbe2e08fda49/users/eb0cc0e78f0240fe82e72939fd3af105/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.279431 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=542e8ff48e7c425eb5c1cbe2e08fda49, user_id=eb0cc0e78f0240fe82e72939fd3af105, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.282571 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.297623 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1385/2766] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:17.301128 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.305503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1382/2767] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:17 2022] PUT /identity/v3/projects/542e8ff48e7c425eb5c1cbe2e08fda49/users/eb0cc0e78f0240fe82e72939fd3af105/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:17.312353 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.357752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.419375 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a524b-e53f-4a17-8532-756da9574bdc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.419661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85adc0e-cbde-46bf-a68d-c0c2ed772d46 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.427771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.428526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.428849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.429004 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.429180 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.429548 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.429901 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.430204 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.430883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.431860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.432511 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.438080 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.445873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1383/2768] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:17.452326 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.476079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a898ab6-7898-4a89-b19a-fd6e18898da0 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.482532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1386/2769] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:17.484463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bead9b-a709-44b3-a376-ec417a9ecc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.485639 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bead9b-a709-44b3-a376-ec417a9ecc28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.486121 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bead9b-a709-44b3-a376-ec417a9ecc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.487269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bead9b-a709-44b3-a376-ec417a9ecc28 admin admin] PATH_INFO: `/v3/projects/542e8ff48e7c425eb5c1cbe2e08fda49/users/eb0cc0e78f0240fe82e72939fd3af105/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.488051 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.491942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bead9b-a709-44b3-a376-ec417a9ecc28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542e8ff48e7c425eb5c1cbe2e08fda49, user_id=eb0cc0e78f0240fe82e72939fd3af105, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.497418 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bead9b-a709-44b3-a376-ec417a9ecc28 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.506033 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef43e8-a838-4cbe-879c-52bdabc14b68 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.516062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1384/2770] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:17 2022] PUT /identity/v3/projects/542e8ff48e7c425eb5c1cbe2e08fda49/users/eb0cc0e78f0240fe82e72939fd3af105/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:17.528960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.529287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.529502 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.567311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] MFA Rules not processed for user `eb0cc0e78f0240fe82e72939fd3af105`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:17.568714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:17.591541 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.592159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.592434 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.592704 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.593356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.594995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.611110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1387/2771] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:17.618125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.632383 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832826f1-d007-4cc9-9df9-557afc9ea091 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.639723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.641017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.641017 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.641313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.642013 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.645047 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.657468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1388/2772] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:17.661666 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.672964 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820dda4-6561-4703-b7ac-98ba1efc0324 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.678621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88827cac-e2c8-4dd9-9db0-d6c50028b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.679332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88827cac-e2c8-4dd9-9db0-d6c50028b20f admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.679609 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88827cac-e2c8-4dd9-9db0-d6c50028b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.679880 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88827cac-e2c8-4dd9-9db0-d6c50028b20f admin admin] PATH_INFO: `/v3/projects/bf17b60aa2b1440385f259d36041dcd2/users/aac8ad827f604c318c19efe5c7763e39/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.682294 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1385/2773] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:17.700853 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:17.703074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:17.708171 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88827cac-e2c8-4dd9-9db0-d6c50028b20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf17b60aa2b1440385f259d36041dcd2, user_id=aac8ad827f604c318c19efe5c7763e39, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:17.713312 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88827cac-e2c8-4dd9-9db0-d6c50028b20f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:17.734927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1389/2774] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:17 2022] PUT /identity/v3/projects/bf17b60aa2b1440385f259d36041dcd2/users/aac8ad827f604c318c19efe5c7763e39/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:17.756075 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.756492 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.756902 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.803008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] MFA Rules not processed for user `aac8ad827f604c318c19efe5c7763e39`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:17.804432 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:17.821822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.823465 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c461c3-8352-4ed2-addb-6658913757f5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:17.912599 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:17.913774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:17.914332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:17.914909 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:17.925219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.937871 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1390/2775] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:17.946490 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.025757 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.028224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.044346 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f66d2d-12fa-4e13-8d4d-4be0ea3bf116 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.044826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1386/2776] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:18.052020 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.052974 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.053764 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.053959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.054973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.058550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.110139 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.111883 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1391/2777] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:17 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:18.112377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:18.119163 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.136747 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62ddd3-0fa3-4af0-84b0-749e03b22053 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.225091 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c57b8-441a-469a-8c8c-165323c2a554 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.233348 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.234315 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.234676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.235332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.235332 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.235567 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.235664 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.235851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.236473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.238270 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:18.239909 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.266172 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1392/2778] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:18 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:18.272687 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.291622 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518cfa2-4b45-412d-8d63-b327d9243d70 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.301918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.302561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.302858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.303153 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.303911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.306302 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.325665 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1393/2779] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:18.329090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.330968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.331679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.343928 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46440846-0d70-4e2e-8efb-4668511867f0 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.345550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1387/2780] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:18.350702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.351435 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.351821 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.351990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] PATH_INFO: `/v3/projects/729657b626644f1aa719c842ff9863c8/users/0a1052bb54c54813943fa565fa75bcbf/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.375126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=729657b626644f1aa719c842ff9863c8, user_id=0a1052bb54c54813943fa565fa75bcbf, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.378566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.381398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.398177 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1394/2781] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:18 2022] PUT /identity/v3/projects/729657b626644f1aa719c842ff9863c8/users/0a1052bb54c54813943fa565fa75bcbf/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:18.404559 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.426877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61310e-7996-43f0-b16b-23927d71df89 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.499857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=100 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.511217 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38767cf6-70c2-47c5-a5cb-166c0a6dea62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.512190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38767cf6-70c2-47c5-a5cb-166c0a6dea62 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.512815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38767cf6-70c2-47c5-a5cb-166c0a6dea62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.513267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38767cf6-70c2-47c5-a5cb-166c0a6dea62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.514343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38767cf6-70c2-47c5-a5cb-166c0a6dea62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.517013 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38767cf6-70c2-47c5-a5cb-166c0a6dea62 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.530317 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1388/2782] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:18.537354 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.550158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59fad4-7cf1-4f7a-8e31-585fe3fc2f6d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.555272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.555946 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.556224 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.556495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] PATH_INFO: `/v3/projects/729657b626644f1aa719c842ff9863c8/users/0a1052bb54c54813943fa565fa75bcbf/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.558516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=729657b626644f1aa719c842ff9863c8, user_id=0a1052bb54c54813943fa565fa75bcbf, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:18.561366 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:18.575459 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1395/2783] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:18 2022] PUT /identity/v3/projects/729657b626644f1aa719c842ff9863c8/users/0a1052bb54c54813943fa565fa75bcbf/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:18.589986 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.590407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.590793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:18.637184 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] MFA Rules not processed for user `0a1052bb54c54813943fa565fa75bcbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:18.638630 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:18.774568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1389/2784] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:18.786077 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:18.895815 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a651fe-4d20-4bfc-b783-30760098de6b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:18.905301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:18.906251 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:18.906546 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:18.906983 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:19.000275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:19.001644 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:19.014322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1396/2785] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:22.801866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:22.889965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:22.904292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b51713-45ae-4f24-bf6d-149191d3416a service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:22.905040 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2214ac-05a3-4dde-854e-a87278606af7 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:22.910209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:22.910921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:22.911210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:22.911515 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] PATH_INFO: `/v3/users/aac8ad827f604c318c19efe5c7763e39` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:22.912761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac8ad827f604c318c19efe5c7763e39)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:22.913890 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:22.914945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:22.915102 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:22.915291 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:22.915662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] PATH_INFO: `/v3/users/3cb3c7ec478b48e1924fe890bf9d26b5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:22.916963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb3c7ec478b48e1924fe890bf9d26b5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:22.920180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:22.990126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac8ad827f604c318c19efe5c7763e39'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:22.991323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac8ad827f604c318c19efe5c7763e39'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:22.996709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb3c7ec478b48e1924fe890bf9d26b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:22.997530 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb3c7ec478b48e1924fe890bf9d26b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:22.998203 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb3c7ec478b48e1924fe890bf9d26b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:23.005695 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac8ad827f604c318c19efe5c7763e39'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:23.008858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb3c7ec478b48e1924fe890bf9d26b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:23.027176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-01dcfae5-de1f-4aa3-b124-43d34dfb427b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb3c7ec478b48e1924fe890bf9d26b5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:23.028131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac8ad827f604c318c19efe5c7763e39'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:23.029870 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-9b9829f5-04e6-4a36-b09c-52c420e89fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac8ad827f604c318c19efe5c7763e39'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:23.047539 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:23.054924 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1390/2786] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:22 2022] DELETE /identity/v3/users/3cb3c7ec478b48e1924fe890bf9d26b5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:23.063051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:23.063810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:23.063975 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:23.068712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1397/2787] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:22 2022] DELETE /identity/v3/users/aac8ad827f604c318c19efe5c7763e39 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:23.075425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7666d-0439-43d2-961c-9285da102015 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:23.075906 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7666d-0439-43d2-961c-9285da102015 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:23.076319 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7666d-0439-43d2-961c-9285da102015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:23.095701 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:23.098035 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:23.099294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:23.108782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-52a7666d-0439-43d2-961c-9285da102015 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:23.111359 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-52a7666d-0439-43d2-961c-9285da102015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:23.187021 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1391/2788] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:23.202533 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:23.204849 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:23.211731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1398/2789] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:23.224882 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7666d-0439-43d2-961c-9285da102015 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:23.226734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-52a7666d-0439-43d2-961c-9285da102015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:23.291433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b8dc3-bd70-4dab-80ca-28dd768e8fb5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:23.295618 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:23.296247 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:23.296547 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:23.296781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:23.309071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:23.310586 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:23.312236 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7666d-0439-43d2-961c-9285da102015 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:23.317742 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:23.318529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:23.318927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:23.319222 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:23.325003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1392/2790] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:23.336506 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:23.339274 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:23.359420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1399/2791] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:23.864846 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:23.867220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:23.872714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:23.876953 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:23.900956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa4731-63f6-4ec5-9183-01bd4786a4f0 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:23.903461 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1a512-f63f-4889-865a-79aed5d7297f service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:23.908606 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:23.908803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:23.909323 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:23.909897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:23.910252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:23.910252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] PATH_INFO: `/v3/projects/bf17b60aa2b1440385f259d36041dcd2` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:23.911158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:23.911158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] PATH_INFO: `/v3/projects/e2ee05fa6cb149f5a1e7a78d8a29249c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:23.911766 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf17b60aa2b1440385f259d36041dcd2)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:23.912653 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ee05fa6cb149f5a1e7a78d8a29249c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:23.914402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:23.926189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:24.013277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ee05fa6cb149f5a1e7a78d8a29249c', 'request_id': 'req-eea06588-d888-4e74-91b9-ebc51d2d3e24'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.016605 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf17b60aa2b1440385f259d36041dcd2', 'request_id': 'req-e76cabaf-a710-4baa-9cb0-d5632d4925d8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.025826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ee05fa6cb149f5a1e7a78d8a29249c', 'request_id': 'req-eea06588-d888-4e74-91b9-ebc51d2d3e24'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.028342 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf17b60aa2b1440385f259d36041dcd2', 'request_id': 'req-e76cabaf-a710-4baa-9cb0-d5632d4925d8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.040039 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ee05fa6cb149f5a1e7a78d8a29249c', 'request_id': 'req-eea06588-d888-4e74-91b9-ebc51d2d3e24'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.043941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf17b60aa2b1440385f259d36041dcd2', 'request_id': 'req-e76cabaf-a710-4baa-9cb0-d5632d4925d8'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.057070 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] The token cache is being invalidate because project e2ee05fa6cb149f5a1e7a78d8a29249c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:24.057631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.058426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.060931 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1393/2792] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:23 2022] DELETE /identity/v3/projects/e2ee05fa6cb149f5a1e7a78d8a29249c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:24.065649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:24.069693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] The token cache is being invalidate because project bf17b60aa2b1440385f259d36041dcd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:24.069693 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.070393 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.072311 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1400/2793] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:23 2022] DELETE /identity/v3/projects/bf17b60aa2b1440385f259d36041dcd2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:24.073105 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:24.076581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:24.078277 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:24.129498 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:24.134597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:24.179112 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cabaf-a710-4baa-9cb0-d5632d4925d8 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:24.185074 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:24.187229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:24.187229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:24.187229 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] PATH_INFO: `/v3/users/0a1052bb54c54813943fa565fa75bcbf` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:24.187770 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1052bb54c54813943fa565fa75bcbf)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:24.187938 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea06588-d888-4e74-91b9-ebc51d2d3e24 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:24.190395 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:24.195392 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:24.196214 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:24.196526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:24.196771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] PATH_INFO: `/v3/users/2b4b3b008d7d404eb4f8e8bf09688c90` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:24.198306 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4b3b008d7d404eb4f8e8bf09688c90)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:24.200531 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:24.253293 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1052bb54c54813943fa565fa75bcbf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.254063 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1052bb54c54813943fa565fa75bcbf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.267142 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1052bb54c54813943fa565fa75bcbf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.274044 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4b3b008d7d404eb4f8e8bf09688c90'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.274854 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4b3b008d7d404eb4f8e8bf09688c90'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.275646 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4b3b008d7d404eb4f8e8bf09688c90'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.288713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4b3b008d7d404eb4f8e8bf09688c90'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.297929 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1052bb54c54813943fa565fa75bcbf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.300231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1052bb54c54813943fa565fa75bcbf'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.311090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4b3b008d7d404eb4f8e8bf09688c90'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.334093 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1401/2794] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:24 2022] DELETE /identity/v3/users/0a1052bb54c54813943fa565fa75bcbf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:24.336562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1394/2795] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:24 2022] DELETE /identity/v3/users/2b4b3b008d7d404eb4f8e8bf09688c90 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:24.722387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:24.724831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:24.738425 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:24.740968 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:24.833560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a29e33-629f-486b-bc39-85bc5162af00 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:24.838458 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:24.839135 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:24.839535 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:24.839996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] PATH_INFO: `/v3/projects/729657b626644f1aa719c842ff9863c8` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:24.841057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] RBAC: Authorizing `identity:delete_project(project_id=729657b626644f1aa719c842ff9863c8)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:24.843133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:24.843333 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd94545-4460-4f7f-ad89-d42e44fdef03 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:24.853649 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:24.854681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:24.855162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:24.855446 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] PATH_INFO: `/v3/projects/09c560ce4aa24d05b2c807ac38e0d7d3` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:24.856823 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c560ce4aa24d05b2c807ac38e0d7d3)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:24.859170 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:24.905584 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:24.910658 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729657b626644f1aa719c842ff9863c8', 'request_id': 'req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.922441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729657b626644f1aa719c842ff9863c8', 'request_id': 'req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.922711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c560ce4aa24d05b2c807ac38e0d7d3', 'request_id': 'req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.928976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:24.931748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c560ce4aa24d05b2c807ac38e0d7d3', 'request_id': 'req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.932378 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729657b626644f1aa719c842ff9863c8', 'request_id': 'req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.942863 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c560ce4aa24d05b2c807ac38e0d7d3', 'request_id': 'req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.944856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] The token cache is being invalidate because project 729657b626644f1aa719c842ff9863c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:24.945200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.945884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-2143dfd8-6bb6-47da-9b2d-b5fe019c823e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.947472 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1395/2796] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:24 2022] DELETE /identity/v3/projects/729657b626644f1aa719c842ff9863c8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:24.953602 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f80bd-60f4-4529-ace1-646c9d665758 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:24.953942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f80bd-60f4-4529-ace1-646c9d665758 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:24.954220 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f80bd-60f4-4529-ace1-646c9d665758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:24.965210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] The token cache is being invalidate because project 09c560ce4aa24d05b2c807ac38e0d7d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:24.968453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.968453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:24.968453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1402/2797] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:24 2022] DELETE /identity/v3/projects/09c560ce4aa24d05b2c807ac38e0d7d3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:24.973196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:24.976049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:24.979619 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-610f80bd-60f4-4529-ace1-646c9d665758 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:24.980869 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-610f80bd-60f4-4529-ace1-646c9d665758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.066186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34c8fb-d23c-4534-b9f6-5938e0d678b4 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.072056 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1396/2798] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:25.077401 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.078071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.078352 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.078628 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] PATH_INFO: `/v3/users/eb0cc0e78f0240fe82e72939fd3af105` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.079924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0cc0e78f0240fe82e72939fd3af105)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.081899 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.085556 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.086189 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.086593 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.116382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:25.118681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.131608 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1397/2799] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:25.139173 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.156567 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0cc0e78f0240fe82e72939fd3af105'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.157041 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12203f8e-3d0e-42fb-a089-8cac223d54ec None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.157474 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0cc0e78f0240fe82e72939fd3af105'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.163457 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.164083 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.164582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.164582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.165253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:25.165558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.167176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.168202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0cc0e78f0240fe82e72939fd3af105'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.181689 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1398/2800] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:25.186190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.191734 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0cc0e78f0240fe82e72939fd3af105'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.193043 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0cc0e78f0240fe82e72939fd3af105'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.197960 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576497c-c5e2-4710-acbd-888d007354bc admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.205324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.206688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.207168 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.207597 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.208463 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.212010 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.224226 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1403/2801] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:24 2022] DELETE /identity/v3/users/eb0cc0e78f0240fe82e72939fd3af105 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:25.262001 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1399/2802] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:25 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:07:25.266069 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.267652 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.375231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b3862-ea2d-4423-aef7-402303139c84 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.383215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:25.458829 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.460144 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.460462 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.460851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.461714 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.463728 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.493310 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1404/2803] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:25 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:25.502197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.520930 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8311cf-6025-46c2-8615-49adeb414e08 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.527314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.528143 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.528436 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.528965 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.530669 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.531956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.544550 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1400/2804] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:25.549597 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.561562 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01af39-ba2e-4bfa-ae61-33cd69632371 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.568662 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496897e-e094-494e-9259-a07204610921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.569054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496897e-e094-494e-9259-a07204610921 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.569420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496897e-e094-494e-9259-a07204610921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.569793 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496897e-e094-494e-9259-a07204610921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.570685 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496897e-e094-494e-9259-a07204610921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.572608 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496897e-e094-494e-9259-a07204610921 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.589161 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1405/2805] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:25.590856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.604565 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b1df-db38-4c6f-99fa-cb91a2e6087a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.611336 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533242-d2f1-428d-93d2-f2869bb86fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.612690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533242-d2f1-428d-93d2-f2869bb86fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.613170 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533242-d2f1-428d-93d2-f2869bb86fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.613648 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533242-d2f1-428d-93d2-f2869bb86fc1 admin admin] PATH_INFO: `/v3/projects/55cf9fe1e278422399672a12bd7c680f/users/60ab69046a5d4d0da5f0a29a788aca96/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.618741 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496897e-e094-494e-9259-a07204610921 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.620681 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-5496897e-e094-494e-9259-a07204610921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.636282 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533242-d2f1-428d-93d2-f2869bb86fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cf9fe1e278422399672a12bd7c680f, user_id=60ab69046a5d4d0da5f0a29a788aca96, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.639054 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496897e-e094-494e-9259-a07204610921 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.641281 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533242-d2f1-428d-93d2-f2869bb86fc1 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.658126 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.660275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.660952 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.661380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] PATH_INFO: `/v3/projects/542e8ff48e7c425eb5c1cbe2e08fda49` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.664601 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=542e8ff48e7c425eb5c1cbe2e08fda49)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:25.667125 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:25.672095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1401/2806] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:25 2022] PUT /identity/v3/projects/55cf9fe1e278422399672a12bd7c680f/users/60ab69046a5d4d0da5f0a29a788aca96/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:25.684902 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.685384 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.685887 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.736203 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542e8ff48e7c425eb5c1cbe2e08fda49', 'request_id': 'req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.742167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] MFA Rules not processed for user `60ab69046a5d4d0da5f0a29a788aca96`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:25.743763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542e8ff48e7c425eb5c1cbe2e08fda49', 'request_id': 'req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.744727 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.752775 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542e8ff48e7c425eb5c1cbe2e08fda49', 'request_id': 'req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.766757 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] The token cache is being invalidate because project 542e8ff48e7c425eb5c1cbe2e08fda49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:25.767065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.767862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-4f5dda4c-9eb4-44be-9fb3-8ae6b6b413b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:25.770464 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1406/2807] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:25 2022] DELETE /identity/v3/projects/542e8ff48e7c425eb5c1cbe2e08fda49 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:25.776389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aead9-78cc-421b-aa2a-b71c8f1367e6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.776760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aead9-78cc-421b-aa2a-b71c8f1367e6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.777055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aead9-78cc-421b-aa2a-b71c8f1367e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.812487 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-bd6aead9-78cc-421b-aa2a-b71c8f1367e6 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:25.814560 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bd6aead9-78cc-421b-aa2a-b71c8f1367e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.827939 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:25.857526 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1402/2808] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:25.873637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.875190 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.917053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1407/2809] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:25.927571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.928409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.928517 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:25.954235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:25.956159 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:25.965032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1408/2810] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:25.972021 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:25.978343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:25.992484 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8edab4-f015-461d-95e7-4cbc0d520881 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:25.998807 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:25.999448 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:25.999723 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:25.999995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] PATH_INFO: `/v3/domains` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.000656 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:07:26.000959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.001685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263d7fd-3e76-4968-839a-fb4b94ba1127 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.003415 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.006581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.007368 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.007649 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.007922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.017434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.019497 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.024235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1409/2811] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:07:26.029130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.034098 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1403/2812] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:26.040374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a96cb-9108-4096-8ae1-38d140575337 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.046443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.047364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.047659 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.048005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.049105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.052418 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.092243 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1410/2813] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:07:26 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:26.099456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.117124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e02f8-fd03-4bec-8788-4ee4bd064772 service cinder] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.203459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.204124 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.204405 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.205873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.205873 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.208033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.235332 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1404/2814] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:07:26 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:07:26.240217 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.253568 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f410a8-93e5-42af-897d-c56c036a68bc admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.259467 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.260265 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.260542 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.260830 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.262154 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.263465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.278114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1411/2815] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:07:26.282950 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.296528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede76a3-cca9-4251-a586-bdb2564fc6fe admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.305129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.305921 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.306344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.306752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.308071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.310219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.329587 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1405/2816] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:07:26.334378 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.334754 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.336497 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:26.346916 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288fd37-6be3-48ab-a771-c1f4cc3b0451 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.355486 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e76dab-0015-423a-935d-5173cfbcf79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.356895 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e76dab-0015-423a-935d-5173cfbcf79b admin admin] REQUEST_METHOD: `PUT` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.358054 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e76dab-0015-423a-935d-5173cfbcf79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.359265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e76dab-0015-423a-935d-5173cfbcf79b admin admin] PATH_INFO: `/v3/projects/bac54ebe93094b019c51cb82c9086533/users/fa8e8f1b6e0c4ca0ae6c210f37f3b67d/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.372571 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821f8ce-82d0-4688-b8c2-08b5c529217b admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.382133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.382787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.383100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.383447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] PATH_INFO: `/v3/users/6fd73a2f6bcd429a98e0a4ed3085e0ac` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.384771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd73a2f6bcd429a98e0a4ed3085e0ac)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.386763 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.388814 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e76dab-0015-423a-935d-5173cfbcf79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac54ebe93094b019c51cb82c9086533, user_id=fa8e8f1b6e0c4ca0ae6c210f37f3b67d, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.395050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e76dab-0015-423a-935d-5173cfbcf79b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.423830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1406/2817] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:07:26 2022] PUT /identity/v3/projects/bac54ebe93094b019c51cb82c9086533/users/fa8e8f1b6e0c4ca0ae6c210f37f3b67d/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:07:26.435842 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.436132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.443592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.473582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd73a2f6bcd429a98e0a4ed3085e0ac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:26.474276 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd73a2f6bcd429a98e0a4ed3085e0ac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:26.485216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd73a2f6bcd429a98e0a4ed3085e0ac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:26.490100 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:26.492786 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:26.508925 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd73a2f6bcd429a98e0a4ed3085e0ac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:26.509840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-07b5ae0d-ac28-4f86-b07e-bf6843abb128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd73a2f6bcd429a98e0a4ed3085e0ac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:26.529746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1412/2818] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:26 2022] DELETE /identity/v3/users/6fd73a2f6bcd429a98e0a4ed3085e0ac => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:26.534973 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.535266 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.535543 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.537132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:26.557048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:26.558175 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:26.603716 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1407/2819] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:26.615051 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.617128 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:26.640561 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1413/2820] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:07:26.651868 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:26.653496 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:26.694727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:26.697738 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e62d-ddea-46e6-b5cf-873699027bb8 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.741483 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623fbe-7e7b-4670-9f3b-016085ddf320 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:26.748841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.749801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.750187 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.750616 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.761431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.762828 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.767482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94a99d-1f40-484c-825c-6a07c257246c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:26.768132 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94a99d-1f40-484c-825c-6a07c257246c service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:26.768416 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94a99d-1f40-484c-825c-6a07c257246c service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:26.768688 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94a99d-1f40-484c-825c-6a07c257246c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:26.773920 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1414/2821] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:26.779033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94a99d-1f40-484c-825c-6a07c257246c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:26.780484 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94a99d-1f40-484c-825c-6a07c257246c service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:26.791974 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1408/2822] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:27.211314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:27.214065 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:27.231088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df729e-67f3-4ca2-bda4-54a99f60a33b service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:27.237444 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:27.238612 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:27.239215 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:27.240191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] PATH_INFO: `/v3/projects/b58ee7c3a21941acad15b0d3dffc504e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:27.242361 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58ee7c3a21941acad15b0d3dffc504e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:27.246570 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:27.355621 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58ee7c3a21941acad15b0d3dffc504e', 'request_id': 'req-6f467555-9764-4b32-be52-f9542ed5dcac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:27.364785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58ee7c3a21941acad15b0d3dffc504e', 'request_id': 'req-6f467555-9764-4b32-be52-f9542ed5dcac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:27.374294 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58ee7c3a21941acad15b0d3dffc504e', 'request_id': 'req-6f467555-9764-4b32-be52-f9542ed5dcac'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:27.396398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] The token cache is being invalidate because project b58ee7c3a21941acad15b0d3dffc504e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:27.396733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:27.397368 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:27.399114 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1415/2823] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:27 2022] DELETE /identity/v3/projects/b58ee7c3a21941acad15b0d3dffc504e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:27.722604 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f761-bb4c-4cad-9039-e79215570be4 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:27.723596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f761-bb4c-4cad-9039-e79215570be4 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:27.723596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f761-bb4c-4cad-9039-e79215570be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:27.757529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7899f761-bb4c-4cad-9039-e79215570be4 None None] MFA Rules not processed for user `fa8e8f1b6e0c4ca0ae6c210f37f3b67d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:27.759256 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7899f761-bb4c-4cad-9039-e79215570be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:27.851225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1409/2824] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:27.868028 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:27.869888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:27.959600 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f467555-9764-4b32-be52-f9542ed5dcac admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:27.968965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:27.968965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:27.968965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:27.968965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:27.980372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:27.982257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:27.994919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1416/2825] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:29.521167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee22b-5bda-4611-8bbc-b5918675b6b0 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:29.522479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee22b-5bda-4611-8bbc-b5918675b6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:29.522479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee22b-5bda-4611-8bbc-b5918675b6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:29.553817 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-a26ee22b-5bda-4611-8bbc-b5918675b6b0 None None] MFA Rules not processed for user `fa8e8f1b6e0c4ca0ae6c210f37f3b67d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:29.555979 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-a26ee22b-5bda-4611-8bbc-b5918675b6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:29.566398 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1410/2826] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:37.993223 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=65,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:39.542730 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:46.852347 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:46.853873 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:46.918630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4db43e-d770-4306-821c-779ece055967 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:46.922343 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:46.922998 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:46.923283 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:46.923556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:46.926049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:46.992811 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:46.994257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:47.004672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1417/2827] 10.222.0.72 () {60 vars in 1396 bytes} [Mon Sep 26 18:07:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:54.801265 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:54.801855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:54.801855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:54.832642 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] MFA Rules not processed for user `fa8e8f1b6e0c4ca0ae6c210f37f3b67d`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:54.834397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:54.844302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1411/2828] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:07:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:56.038990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:56.050959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee53dc3-a5f5-43db-b09d-5b0d6df80d07 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:56.057526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:56.058152 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:56.058433 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:56.058711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:56.060733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:56.143648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:56.145495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff81804-7516-49ae-a518-dc3c17c2512e service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:56.157941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1418/2829] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:03.412903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:03.415474 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:03.445460 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cff75-96f1-497a-8463-d58faba8e584 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:03.453036 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:03.454219 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:03.454677 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:03.456053 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] PATH_INFO: `/v3/users/60ab69046a5d4d0da5f0a29a788aca96` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:03.459700 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ab69046a5d4d0da5f0a29a788aca96)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:03.463235 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:03.535734 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ab69046a5d4d0da5f0a29a788aca96'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:03.536581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ab69046a5d4d0da5f0a29a788aca96'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:03.537379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ab69046a5d4d0da5f0a29a788aca96'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:03.546153 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ab69046a5d4d0da5f0a29a788aca96'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:03.561635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ab69046a5d4d0da5f0a29a788aca96'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:03.581847 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1412/2830] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:08:03 2022] DELETE /identity/v3/users/60ab69046a5d4d0da5f0a29a788aca96 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:03.586147 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:03.586661 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:03.586818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:03.622364 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:03.623976 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:03.700400 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:03.741486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1419/2831] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:03.762780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:03.766663 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:03.875415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66947c-9c97-4319-a86b-1af53055cfce admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:03.882956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640c69a-9a17-4532-9e94-369183d6e976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:03.883601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640c69a-9a17-4532-9e94-369183d6e976 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:03.883903 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640c69a-9a17-4532-9e94-369183d6e976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:03.884171 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640c69a-9a17-4532-9e94-369183d6e976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:03.896948 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640c69a-9a17-4532-9e94-369183d6e976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:03.898305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640c69a-9a17-4532-9e94-369183d6e976 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:03.916106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1413/2832] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:08:04.310516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:04.313451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:04.336307 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448138e0-1ed2-49b2-aadf-a1d23d86fc7f None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:04.346457 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:04.347869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:04.348458 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:04.350060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] PATH_INFO: `/v3/projects/55cf9fe1e278422399672a12bd7c680f` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:04.353436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cf9fe1e278422399672a12bd7c680f)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:04.358272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:04.470765 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cf9fe1e278422399672a12bd7c680f', 'request_id': 'req-1ef894ff-28d8-4f07-bb18-5d722490fdce'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:04.481027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cf9fe1e278422399672a12bd7c680f', 'request_id': 'req-1ef894ff-28d8-4f07-bb18-5d722490fdce'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:04.490921 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cf9fe1e278422399672a12bd7c680f', 'request_id': 'req-1ef894ff-28d8-4f07-bb18-5d722490fdce'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:04.538237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] The token cache is being invalidate because project 55cf9fe1e278422399672a12bd7c680f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:08:04.539324 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:04.541022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-1ef894ff-28d8-4f07-bb18-5d722490fdce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:04.544071 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1420/2833] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:08:04 2022] DELETE /identity/v3/projects/55cf9fe1e278422399672a12bd7c680f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:04.553564 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:04.557127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:04.557963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:04.628085 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:04.629844 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:04.753432 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1414/2834] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:04.762304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:04.763162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:04.763162 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:04.818191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:04.821289 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:04.831696 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1421/2835] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:04.846540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:04.879855 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96e6f2-d559-44eb-b19d-eed4e7b103ea None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:04.889389 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:04.890733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:04.891266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:04.891810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:04.892908 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:08:04.893471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:04.895369 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:04.899068 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:04.938377 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1415/2836] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:08:04.939648 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:04.958196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9a444-85e4-47c5-8f04-73959b161b58 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:04.983023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:04.983557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:04.983858 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:04.984105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:04.984773 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:04.986914 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:05.066916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1422/2837] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:08:04 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:08:05.078243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:05.094860 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db1788-d167-485a-b549-4e4b4438b551 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:05.177785 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:05.178815 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:05.179267 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:05.179551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:05.180198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:05.182176 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:05.209224 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1416/2838] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:08:05 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:08:05.213785 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:05.229557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb5d72-e510-4825-a9ea-e88e51e84683 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:05.237290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:05.237859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:05.238134 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:05.238403 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:05.239313 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:05.240963 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:05.261782 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1423/2839] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:08:05.265487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:05.280031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f5541-fc01-4616-9106-35383529209d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:05.286180 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c42ad5-86ee-4ddc-b920-cc8a60a64516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:05.289926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42ad5-86ee-4ddc-b920-cc8a60a64516 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:05.290357 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42ad5-86ee-4ddc-b920-cc8a60a64516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:05.290750 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42ad5-86ee-4ddc-b920-cc8a60a64516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:05.291495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c42ad5-86ee-4ddc-b920-cc8a60a64516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:05.293215 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c42ad5-86ee-4ddc-b920-cc8a60a64516 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:05.316057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1417/2840] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:08:05.320431 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:05.334199 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983fe6-0a32-464e-95f0-04400b170188 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:05.339705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:05.340357 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:05.340690 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:05.340918 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] PATH_INFO: `/v3/projects/21586a671de94dc9b7b73f5dbaac6a01/users/4f51a306feb34c33b3e9162d8253ae38/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:05.365748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21586a671de94dc9b7b73f5dbaac6a01, user_id=4f51a306feb34c33b3e9162d8253ae38, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:05.368695 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:05.385991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1424/2841] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:08:05 2022] PUT /identity/v3/projects/21586a671de94dc9b7b73f5dbaac6a01/users/4f51a306feb34c33b3e9162d8253ae38/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:05.390963 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:05.391253 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:05.395976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:05.426893 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:05.428487 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:05.558006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1418/2842] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:05.570875 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:05.573053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:05.686750 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4805a-c489-4b50-8f1e-2c4cfdab16e3 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:05.692202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:05.693011 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:05.693309 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:05.693564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:05.708834 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:05.711526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:05.735111 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1425/2843] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:06.101765 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:06.104516 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:06.308668 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:06.312552 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:06.361317 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a883f6-e3e4-4f20-ad37-bbd443f1f48d service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:06.369629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:06.370165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:06.370445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:06.370718 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] PATH_INFO: `/v3/users/13d1363fdfcf4d889f1ab927310e0374` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:06.371901 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d1363fdfcf4d889f1ab927310e0374)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:06.373820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:06.376230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff77dd7-1c35-4f8c-8745-ac88109a99c5 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:06.382157 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:06.406479 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:06.407312 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:06.407756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:06.408221 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:06.414807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:06.453495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:06.460049 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:06.485576 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d1363fdfcf4d889f1ab927310e0374'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:06.487053 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d1363fdfcf4d889f1ab927310e0374'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:06.502076 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1419/2844] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:06.513048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d1363fdfcf4d889f1ab927310e0374'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:06.545861 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d1363fdfcf4d889f1ab927310e0374'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:06.546588 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-e86f6800-a1a5-4eca-b311-d59c8c4e9ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d1363fdfcf4d889f1ab927310e0374'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:06.570818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1426/2845] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:08:06 2022] DELETE /identity/v3/users/13d1363fdfcf4d889f1ab927310e0374 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:06.806686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:06.819793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:06.832310 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c2e2-e9ec-4233-839a-de77bd37eec7 service placement] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:06.940779 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:06.941614 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:06.941978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:06.942333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] PATH_INFO: `/v3/projects/7fc64ff05f7240b3a9cb0f191195b8e3` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:06.943711 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc64ff05f7240b3a9cb0f191195b8e3)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:06.945857 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:07.021721 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:07.022105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:07.022453 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:07.022698 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc64ff05f7240b3a9cb0f191195b8e3', 'request_id': 'req-218b0f18-778a-4552-88e0-0c266c669ee5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.030737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc64ff05f7240b3a9cb0f191195b8e3', 'request_id': 'req-218b0f18-778a-4552-88e0-0c266c669ee5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.041568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc64ff05f7240b3a9cb0f191195b8e3', 'request_id': 'req-218b0f18-778a-4552-88e0-0c266c669ee5'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.058158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] The token cache is being invalidate because project 7fc64ff05f7240b3a9cb0f191195b8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:08:07.058471 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.059482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.061972 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1420/2846] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:08:06 2022] DELETE /identity/v3/projects/7fc64ff05f7240b3a9cb0f191195b8e3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:07.063660 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] MFA Rules not processed for user `4f51a306feb34c33b3e9162d8253ae38`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:07.065564 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:07.067881 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:07.071361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:07.178725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b0f18-778a-4552-88e0-0c266c669ee5 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:07.187327 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:07.188158 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:07.194400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:07.194400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] PATH_INFO: `/v3/users/a63f0f59c26a4bd98743f0eaaf7bd94c` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:07.194826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1427/2847] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:08:07.195046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63f0f59c26a4bd98743f0eaaf7bd94c)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:07.195046 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:07.221629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:07.223445 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:07.260300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:07.275504 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63f0f59c26a4bd98743f0eaaf7bd94c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.277760 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63f0f59c26a4bd98743f0eaaf7bd94c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.279485 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63f0f59c26a4bd98743f0eaaf7bd94c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.288225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63f0f59c26a4bd98743f0eaaf7bd94c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.304011 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63f0f59c26a4bd98743f0eaaf7bd94c'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.310508 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088c35c-75d1-4ffa-b2b5-bd829e6a55a8 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:07.315372 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:07.316420 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:07.316831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:07.317941 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:07.319443 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:07.323877 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1421/2848] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:08:07 2022] DELETE /identity/v3/users/a63f0f59c26a4bd98743f0eaaf7bd94c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:08:07.332544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:07.334003 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975e9e7-6858-487c-9f05-870fa3821e14 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:07.347893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1428/2849] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:08:07.661303 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:07.665225 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:07.791429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e2698-0325-44ab-841b-2993b6204fca admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:07.799440 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:07.800304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:07.800744 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:07.801197 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] PATH_INFO: `/v3/projects/2b32003d56574fe082f58431b531a350` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:07.802741 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b32003d56574fe082f58431b531a350)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:07.805925 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:07.888749 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b32003d56574fe082f58431b531a350', 'request_id': 'req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.898748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b32003d56574fe082f58431b531a350', 'request_id': 'req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.907637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b32003d56574fe082f58431b531a350', 'request_id': 'req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.921092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] The token cache is being invalidate because project 2b32003d56574fe082f58431b531a350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:08:07.921382 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.922133 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0473b7eb-aef4-4dd1-aeea-d7305e4e3f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:07.923737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1422/2850] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:08:07 2022] DELETE /identity/v3/projects/2b32003d56574fe082f58431b531a350 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:08.879320 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:08.879840 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:08.879965 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:08.906391 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] MFA Rules not processed for user `4f51a306feb34c33b3e9162d8253ae38`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:08.907827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:08.920555 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:08.995301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1429/2851] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:08:16.462415 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa62afc-6ea7-4416-a4dd-87f9520e83e6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.463014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa62afc-6ea7-4416-a4dd-87f9520e83e6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.463014 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa62afc-6ea7-4416-a4dd-87f9520e83e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.498558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-9aa62afc-6ea7-4416-a4dd-87f9520e83e6 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:16.499940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-9aa62afc-6ea7-4416-a4dd-87f9520e83e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:16.538413 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:16.570515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1423/2852] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:16.637733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:16.639550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:16.721797 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0983cd-09c0-4f71-9b5d-4b0d559496c5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:16.727527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:16.728186 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.728465 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.728737 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.738308 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:16.739959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:16.749642 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1430/2853] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:08:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:08:26.559355 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=9 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:26.745290 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=50 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:36.630116 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:36.630635 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:36.630906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:36.657693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:36.660620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:36.672012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1424/2854] 10.222.0.72 () {58 vars in 967 bytes} [Mon Sep 26 18:08:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:36.685386 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:36.701425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a8356-9739-4fbe-99d4-18d0aa6b8422 service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:36.709206 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:36.709825 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:36.710105 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:36.710380 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:36.720060 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:36.722407 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:36.733992 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1431/2855] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:08:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:08:38.943208 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:38.946560 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:39.027300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bce0f-596a-45eb-b549-0b5cb4d86c0f None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.033978 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:39.034774 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.035347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.035662 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] PATH_INFO: `/v3/users/fa8e8f1b6e0c4ca0ae6c210f37f3b67d` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.036839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8e8f1b6e0c4ca0ae6c210f37f3b67d)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:39.038781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:39.092482 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8e8f1b6e0c4ca0ae6c210f37f3b67d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.093118 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8e8f1b6e0c4ca0ae6c210f37f3b67d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.093767 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8e8f1b6e0c4ca0ae6c210f37f3b67d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.101683 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8e8f1b6e0c4ca0ae6c210f37f3b67d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.118866 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-42e26379-f3c0-4653-823f-d57a12f64363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8e8f1b6e0c4ca0ae6c210f37f3b67d'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.134899 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1425/2856] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:08:38 2022] DELETE /identity/v3/users/fa8e8f1b6e0c4ca0ae6c210f37f3b67d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:39.334991 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:39.346369 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638488f-6867-4d09-afc5-a3f42bbcae36 service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.430581 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:39.431556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.431556 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.431999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] PATH_INFO: `/v3/projects/bac54ebe93094b019c51cb82c9086533` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.435399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac54ebe93094b019c51cb82c9086533)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:39.435399 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:39.493613 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac54ebe93094b019c51cb82c9086533', 'request_id': 'req-695f7659-883e-4c02-bde1-698647e4da1f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.500831 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac54ebe93094b019c51cb82c9086533', 'request_id': 'req-695f7659-883e-4c02-bde1-698647e4da1f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.512871 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac54ebe93094b019c51cb82c9086533', 'request_id': 'req-695f7659-883e-4c02-bde1-698647e4da1f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.528532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] The token cache is being invalidate because project bac54ebe93094b019c51cb82c9086533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:08:39.528894 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.529968 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-695f7659-883e-4c02-bde1-698647e4da1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:39.532425 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1432/2857] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:08:39 2022] DELETE /identity/v3/projects/bac54ebe93094b019c51cb82c9086533 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:39.540609 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.540923 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.541238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.570390 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:39.571652 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:39.647922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1426/2858] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:39.655919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.656532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.657022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.680596 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:39.681888 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:39.690583 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1433/2859] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:39.700709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:39.713276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72e93b-8e7e-4ebd-9f82-4f399021df55 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.718228 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:39.718872 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.719154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.719429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.720088 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:08:39.720397 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:39.723222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:39.733475 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:39.738322 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1427/2860] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:08:39.745869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:39.756130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30c768-9249-4dc1-9c23-55a41ea31cee None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.761787 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:39.762643 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.763106 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.763394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.764356 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:39.768447 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:39.810146 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1434/2861] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:08:39 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:08:39.816333 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:39.827192 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cada7e-f550-4738-ac84-e60847fa0bc9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.900207 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:39.900897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.901079 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.901362 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.902043 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:39.904210 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:39.934861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1428/2862] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:08:39 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:08:39.941604 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:39.953703 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac59759-e6a0-40b9-a8ca-5247b85fb717 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.958733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:39.959384 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:39.959664 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:39.959961 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:39.960615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:39.962213 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:39.974468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1435/2863] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:08:39.978645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:39.991977 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436778a-3a02-4e4d-ad9c-2f4decf753e2 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:39.999205 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb31c0a-505f-4e02-9d3d-cf22a12a8b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:40.000090 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31c0a-505f-4e02-9d3d-cf22a12a8b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:40.000518 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31c0a-505f-4e02-9d3d-cf22a12a8b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:40.000926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31c0a-505f-4e02-9d3d-cf22a12a8b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:40.002713 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb31c0a-505f-4e02-9d3d-cf22a12a8b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:40.005804 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb31c0a-505f-4e02-9d3d-cf22a12a8b36 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:40.025037 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1429/2864] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:08:40.030951 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:40.043634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bcf52-eed0-494a-92f3-68bd7b88446e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:40.049986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:40.050895 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:40.051232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:40.051573 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] PATH_INFO: `/v3/projects/6a03384ef2f043ecaa9c1caaced113a7/users/b26e05a7dfa2449a9b0aed3cbe131ba0/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:40.073113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a03384ef2f043ecaa9c1caaced113a7, user_id=b26e05a7dfa2449a9b0aed3cbe131ba0, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:40.078500 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:40.098023 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1436/2865] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:08:40 2022] PUT /identity/v3/projects/6a03384ef2f043ecaa9c1caaced113a7/users/b26e05a7dfa2449a9b0aed3cbe131ba0/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:40.112982 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:40.113313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:40.113528 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:40.145784 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] MFA Rules not processed for user `b26e05a7dfa2449a9b0aed3cbe131ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:40.147071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:40.235661 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1430/2866] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:08:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:08:40.250512 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:40.252798 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:40.271940 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:40.327633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04e0e1-72c8-41ff-a494-a517001dff8f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:40.332841 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:40.333473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:40.333756 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:40.334031 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:40.343629 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:40.345113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:40.356450 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1437/2867] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:50.231097 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:50.351836 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=20 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:54.769141 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:54.770735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:54.851223 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aede9be-fab5-49b2-ad42-0f9143e79290 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:54.856101 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:54.856790 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:54.857117 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:54.857441 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:54.859795 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:54.946576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:54.948074 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:54.982349 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1431/2868] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:59.957139 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:59.968143 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b1058-24e2-4c71-8a81-fe484fe268c4 service cinder] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:59.972846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:59.974218 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:59.974711 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:59.975237 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:59.980252 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:00.001070 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:00.003168 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2f727-503f-4d3c-9ff7-8a3bd6ad0af8 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:00.016632 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1438/2869] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:09:04.702733 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:04.789793 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab5561-d44d-43d2-bc9e-f22569f25e24 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:04.795230 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:04.795995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:04.796364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:04.796736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] PATH_INFO: `/v3/users/b26e05a7dfa2449a9b0aed3cbe131ba0` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:04.798000 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26e05a7dfa2449a9b0aed3cbe131ba0)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:04.800298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:04.860809 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:04.864257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26e05a7dfa2449a9b0aed3cbe131ba0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:04.865385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26e05a7dfa2449a9b0aed3cbe131ba0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:04.866234 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26e05a7dfa2449a9b0aed3cbe131ba0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:04.873883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26e05a7dfa2449a9b0aed3cbe131ba0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:04.888568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26e05a7dfa2449a9b0aed3cbe131ba0'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:04.904775 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1432/2870] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:09:04 2022] DELETE /identity/v3/users/b26e05a7dfa2449a9b0aed3cbe131ba0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:04.909510 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:04.909995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:04.910140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:04.929390 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:04.930519 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:04.999140 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1439/2871] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:09:05.009825 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:05.011346 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:05.096668 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595edb0-b748-4d13-9513-ef7e11dda505 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:05.101581 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661ff8-a148-4530-a610-97b7deb16fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:05.102209 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661ff8-a148-4530-a610-97b7deb16fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.102666 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661ff8-a148-4530-a610-97b7deb16fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.102996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661ff8-a148-4530-a610-97b7deb16fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.115605 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661ff8-a148-4530-a610-97b7deb16fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:05.117287 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661ff8-a148-4530-a610-97b7deb16fb2 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:05.137279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1433/2872] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:05.498725 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:05.500842 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:05.512934 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3ce15-e0f2-42f1-a778-488dc40978e5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:05.518329 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:05.518985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.519529 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.519806 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] PATH_INFO: `/v3/projects/6a03384ef2f043ecaa9c1caaced113a7` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.520985 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a03384ef2f043ecaa9c1caaced113a7)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:05.522926 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:05.580702 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a03384ef2f043ecaa9c1caaced113a7', 'request_id': 'req-04381f5a-06ff-4baa-8bbd-3b9c090696a6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:05.587049 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a03384ef2f043ecaa9c1caaced113a7', 'request_id': 'req-04381f5a-06ff-4baa-8bbd-3b9c090696a6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:05.594024 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a03384ef2f043ecaa9c1caaced113a7', 'request_id': 'req-04381f5a-06ff-4baa-8bbd-3b9c090696a6'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:05.608915 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] The token cache is being invalidate because project 6a03384ef2f043ecaa9c1caaced113a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:09:05.609211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:05.609869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-04381f5a-06ff-4baa-8bbd-3b9c090696a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:05.612214 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1440/2873] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:09:05 2022] DELETE /identity/v3/projects/6a03384ef2f043ecaa9c1caaced113a7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:05.617238 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f477c9-055a-4942-babf-d1c382389165 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.617622 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f477c9-055a-4942-babf-d1c382389165 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.617897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f477c9-055a-4942-babf-d1c382389165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.639172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-46f477c9-055a-4942-babf-d1c382389165 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:05.640399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-46f477c9-055a-4942-babf-d1c382389165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:05.710897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1434/2874] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:05.720306 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.721292 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.721532 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.743522 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:05.744760 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:05.753131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1441/2875] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:05.757735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f477c9-055a-4942-babf-d1c382389165 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:05.770976 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f477c9-055a-4942-babf-d1c382389165 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:05.775856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:05.776676 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.776861 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.777131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.777949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:09:05.778252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:05.779918 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:05.800865 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1435/2876] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:09:05.805544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:05.816304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b2624-6d7c-436f-9955-94557fd27cc3 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:05.821304 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:05.821897 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.822178 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.822449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.823148 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:05.825048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:05.872027 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1442/2877] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:09:05 2022] POST /identity/v3/projects => generated 397 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:09:05.876515 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:05.890167 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0dfdd-0751-43d1-b2c7-16026d0c70d9 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:05.943071 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:05.956542 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:05.957157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.957438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.957714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.958347 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:05.960331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:05.990540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1436/2878] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:09:05 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:09:05.995244 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:06.007235 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c012-b142-404d-bd70-946f52616ead admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:06.014301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:06.015389 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:06.015919 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:06.016436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:06.018209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:06.021544 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:06.036849 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1443/2879] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:06.041254 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:06.051839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cc183-0a65-4256-a778-59d0a9a8db75 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:06.057279 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7137f47-1913-4c9d-8666-e4f40793f0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:06.057906 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7137f47-1913-4c9d-8666-e4f40793f0ec admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:06.058218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7137f47-1913-4c9d-8666-e4f40793f0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:06.058495 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7137f47-1913-4c9d-8666-e4f40793f0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:06.059196 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7137f47-1913-4c9d-8666-e4f40793f0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:06.060889 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7137f47-1913-4c9d-8666-e4f40793f0ec admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:06.076596 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1437/2880] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:06.081987 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:06.093370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de737c2a-87e0-43a1-8e50-97fdb2f59233 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:06.099216 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:06.099949 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:06.100250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:06.100551 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] PATH_INFO: `/v3/projects/17f3e418572845cc87b19935fc91de23/users/c76ad717a2ab431e89ef0d9b9a320e60/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:06.120518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f3e418572845cc87b19935fc91de23, user_id=c76ad717a2ab431e89ef0d9b9a320e60, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:06.125674 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:06.138865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:06.149316 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1444/2881] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:09:06 2022] PUT /identity/v3/projects/17f3e418572845cc87b19935fc91de23/users/c76ad717a2ab431e89ef0d9b9a320e60/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:06.162673 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:06.162994 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:06.163266 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:06.210060 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] MFA Rules not processed for user `c76ad717a2ab431e89ef0d9b9a320e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:06.211364 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:06.292913 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1438/2882] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:06.306078 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:06.307812 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:06.377735 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c3b07-3e9b-472e-8749-1eb2a397140d admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:06.381771 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278e407-99cc-4fbe-a734-09c55444ccba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39bde61b0b7c4cf3950ef63248a3b0da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:06.382402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278e407-99cc-4fbe-a734-09c55444ccba service cinder] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:06.382679 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278e407-99cc-4fbe-a734-09c55444ccba service cinder] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:06.382990 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278e407-99cc-4fbe-a734-09c55444ccba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:06.392516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278e407-99cc-4fbe-a734-09c55444ccba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:06.394088 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278e407-99cc-4fbe-a734-09c55444ccba service cinder] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:06.403553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1445/2883] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:15.863720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:15.979246 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092c382-d658-473c-a5cc-46a09c8844ee None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:15.992277 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:15.993544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:15.994065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:15.994585 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] PATH_INFO: `/v3/users/c76ad717a2ab431e89ef0d9b9a320e60` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:15.996839 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76ad717a2ab431e89ef0d9b9a320e60)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:16.000791 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:16.069342 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76ad717a2ab431e89ef0d9b9a320e60'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.070351 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76ad717a2ab431e89ef0d9b9a320e60'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.071324 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76ad717a2ab431e89ef0d9b9a320e60'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.085456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76ad717a2ab431e89ef0d9b9a320e60'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.092644 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:16.107926 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76ad717a2ab431e89ef0d9b9a320e60'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.134693 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1439/2884] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:09:15 2022] DELETE /identity/v3/users/c76ad717a2ab431e89ef0d9b9a320e60 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:16.140210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:16.140851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:16.140851 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:16.169006 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:16.171777 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:16.282404 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1446/2885] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:16.295106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:16.297620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:16.395625 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3083f-7c59-49ad-938e-93f6a1e9d2d8 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:16.399884 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef375ee1-cda4-42cb-b722-380a076d0094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:16.400737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375ee1-cda4-42cb-b722-380a076d0094 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:16.401252 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375ee1-cda4-42cb-b722-380a076d0094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:16.401756 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375ee1-cda4-42cb-b722-380a076d0094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:16.412996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef375ee1-cda4-42cb-b722-380a076d0094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:16.414725 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef375ee1-cda4-42cb-b722-380a076d0094 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:16.427708 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1440/2886] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:16.788230 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:16.789744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:16.802889 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a70b-b658-4b04-8cf1-e0b7136c38d5 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:16.811331 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:16.811995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:16.812282 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:16.812553 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] PATH_INFO: `/v3/projects/17f3e418572845cc87b19935fc91de23` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:16.813789 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f3e418572845cc87b19935fc91de23)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:16.817367 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:16.889062 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f3e418572845cc87b19935fc91de23', 'request_id': 'req-5060cb80-a77d-412e-8748-48e1d9dd7634'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.901758 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f3e418572845cc87b19935fc91de23', 'request_id': 'req-5060cb80-a77d-412e-8748-48e1d9dd7634'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.909032 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f3e418572845cc87b19935fc91de23', 'request_id': 'req-5060cb80-a77d-412e-8748-48e1d9dd7634'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.927408 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] The token cache is being invalidate because project 17f3e418572845cc87b19935fc91de23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:09:16.927701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.928360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-5060cb80-a77d-412e-8748-48e1d9dd7634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:16.931181 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1447/2887] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:09:16 2022] DELETE /identity/v3/projects/17f3e418572845cc87b19935fc91de23 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:16.936033 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:16.937299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:16.937735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:16.979611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:16.980856 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:17.079276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1441/2888] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:17.089893 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.090358 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.090924 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.115746 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:17.116984 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:17.125618 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1448/2889] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:09:17.131003 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.141996 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebbe5e-d3d2-47f1-bf46-38fc17a40aea None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.148470 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.149092 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.149371 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.149645 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] PATH_INFO: `/v3/domains` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.150298 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:09:17.150601 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.152243 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.166311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1442/2890] 10.222.0.72 () {58 vars in 1153 bytes} [Mon Sep 26 18:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:09:17.174398 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.186068 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21d0d6-a764-47ea-a851-c93a10554193 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.191948 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.192866 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.193220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.193634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.194748 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.196923 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.216037 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:17.238916 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1449/2891] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:09:17 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:09:17.240603 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.253200 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fd5f1-9acd-4f8f-863e-c1c8aee0007a admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.344172 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.344995 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.345299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.346828 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] PATH_INFO: `/v3/users` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.347576 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.349721 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.374631 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:17.395127 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1443/2892] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:09:17 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:09:17.396726 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.410473 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40773ff1-1f02-4835-befd-4d7645859ea7 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.416582 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.417527 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.417795 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.418130 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.418801 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.420731 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.435073 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1450/2893] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:17.440264 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.463522 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7734edb-f844-4cb3-b142-e49201662b52 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.484770 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3b494-148d-4275-9f6d-25ab20e9580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.485520 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3b494-148d-4275-9f6d-25ab20e9580b admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.486006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3b494-148d-4275-9f6d-25ab20e9580b admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.488594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3b494-148d-4275-9f6d-25ab20e9580b admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.488594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3b494-148d-4275-9f6d-25ab20e9580b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.490686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3b494-148d-4275-9f6d-25ab20e9580b admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.510299 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1444/2894] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:17.515832 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.530016 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee0059-76f0-4c97-a134-c4b0e2ecd415 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.539585 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.540279 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.540554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.540827 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] PATH_INFO: `/v3/projects/c747c13a887a4f45bf8a940833be3831/users/f95b1dd078c842df97f8fb242e59020b/roles/7642cb8c018c4602b0a15c0e67781890` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.561701 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c747c13a887a4f45bf8a940833be3831, user_id=f95b1dd078c842df97f8fb242e59020b, role_id=7642cb8c018c4602b0a15c0e67781890)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.566675 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.591860 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1451/2895] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:09:17 2022] PUT /identity/v3/projects/c747c13a887a4f45bf8a940833be3831/users/f95b1dd078c842df97f8fb242e59020b/roles/7642cb8c018c4602b0a15c0e67781890 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:09:17.598940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.598940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.598940 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.624006 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] MFA Rules not processed for user `91cdfaf440314ad2b1f94da968a2f0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:17.625537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:17.702919 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1445/2896] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:09:17.727220 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:17.729092 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:17.822300 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c7a45-70d4-48f0-9507-fb1480bbb3be admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:17.826523 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e888d5a-8d7b-4842-ab43-28429d8dafc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:17.827191 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888d5a-8d7b-4842-ab43-28429d8dafc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:17.827471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888d5a-8d7b-4842-ab43-28429d8dafc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:17.827808 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e888d5a-8d7b-4842-ab43-28429d8dafc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:17.840318 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e888d5a-8d7b-4842-ab43-28429d8dafc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:17.841010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e888d5a-8d7b-4842-ab43-28429d8dafc9 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:17.852818 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1452/2897] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:09:19.157426 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:19.162735 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:19.175674 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268055c-b74c-46e1-bf20-1e645ed2b5fb None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:19.179762 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:19.180537 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:19.180922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:19.181300 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:19.184062 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:19.200015 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:19.201883 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:19.215473 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1446/2898] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:24.262387 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.263166 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.263524 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.324803 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] MFA Rules not processed for user `f95b1dd078c842df97f8fb242e59020b`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:24.326232 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:24.442481 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1453/2899] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:24.446714 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.458023 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f51f9-6c68-44e2-9d5a-5fcccbf73741 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.463285 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.464012 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.464379 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.464719 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.465446 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.467541 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.504257 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1447/2900] 10.222.0.72 () {60 vars in 1153 bytes} [Mon Sep 26 18:09:24 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:09:24.508509 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.523322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8456a0a-d914-4611-ba41-90d077c61c89 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.596273 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:24.608239 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.609495 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.610005 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.610488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] PATH_INFO: `/v3/users` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.611449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.613580 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.638272 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1454/2901] 10.222.0.72 () {60 vars in 1144 bytes} [Mon Sep 26 18:09:24 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:09:24.643241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.657399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e17f-4193-402b-99d4-67182d7f1908 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.666057 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.667314 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.667897 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.668469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.669709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.672922 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.692453 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1448/2902] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:24.696488 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.708577 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca113b-10ca-4f26-9782-3d9011454401 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.713672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.714607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.714927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.715210 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] PATH_INFO: `/v3/projects/99ac71f2187a4db1be4f404fa3cf04a5/users/2e323c9ad9b3491496d8023fed2731b4/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.746574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ac71f2187a4db1be4f404fa3cf04a5, user_id=2e323c9ad9b3491496d8023fed2731b4, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.749394 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.768344 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1455/2903] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:09:24 2022] PUT /identity/v3/projects/99ac71f2187a4db1be4f404fa3cf04a5/users/2e323c9ad9b3491496d8023fed2731b4/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:09:24.772448 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.787302 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e49d8f-3c4a-4ad4-b5fd-590bcecaa90d admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.855943 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05357ff-2070-4a74-adb3-eaef3e81e405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.856562 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05357ff-2070-4a74-adb3-eaef3e81e405 admin admin] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.856837 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05357ff-2070-4a74-adb3-eaef3e81e405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.857106 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05357ff-2070-4a74-adb3-eaef3e81e405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.857748 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05357ff-2070-4a74-adb3-eaef3e81e405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.859433 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05357ff-2070-4a74-adb3-eaef3e81e405 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.870771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1449/2904] 10.222.0.72 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:24.876234 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.888615 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a8eee-6d77-43d3-9156-fe16f039e914 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.895563 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.896133 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.896514 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.896886 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] PATH_INFO: `/v3/projects/99ac71f2187a4db1be4f404fa3cf04a5/users/2e323c9ad9b3491496d8023fed2731b4/roles/3c63afd56d04499eb8c3c56d8aa62dba` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.900550 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ac71f2187a4db1be4f404fa3cf04a5, user_id=2e323c9ad9b3491496d8023fed2731b4, role_id=3c63afd56d04499eb8c3c56d8aa62dba)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.902055 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.918518 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1456/2905] 10.222.0.72 () {60 vars in 1483 bytes} [Mon Sep 26 18:09:24 2022] PUT /identity/v3/projects/99ac71f2187a4db1be4f404fa3cf04a5/users/2e323c9ad9b3491496d8023fed2731b4/roles/3c63afd56d04499eb8c3c56d8aa62dba => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:30.349456 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:30.350722 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:30.351296 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:30.381480 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] MFA Rules not processed for user `4f51a306feb34c33b3e9162d8253ae38`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:30.383241 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:30.438222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:30.473434 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1450/2906] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:30.541201 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:30.547291 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:30.726986 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cd904-ce8e-4d42-82b7-d7c095845c84 admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:30.733058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:30.733727 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:30.734096 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:30.734370 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] PATH_INFO: `/v3/users/9979be70f604408d9b78050e3ba52f6e` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:30.736463 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9979be70f604408d9b78050e3ba52f6e)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:30.739633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:30.848698 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9979be70f604408d9b78050e3ba52f6e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:30.854121 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9979be70f604408d9b78050e3ba52f6e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:30.877557 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9979be70f604408d9b78050e3ba52f6e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:30.896566 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9979be70f604408d9b78050e3ba52f6e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:30.897468 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9979be70f604408d9b78050e3ba52f6e'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:30.922676 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1457/2907] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:09:30 2022] DELETE /identity/v3/users/9979be70f604408d9b78050e3ba52f6e => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:31.182769 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:31.196709 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613a6fe-09d8-435e-b91e-24c3dc3d091c None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:31.284971 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:31.286130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:31.286130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:31.286459 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] PATH_INFO: `/v3/projects/9d0160f2201546a385be13aea0454fdc` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:31.288199 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0160f2201546a385be13aea0454fdc)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:31.290830 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:31.356385 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0160f2201546a385be13aea0454fdc', 'request_id': 'req-53481889-cbd6-4aa7-9df4-9d882ee48b13'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.366826 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0160f2201546a385be13aea0454fdc', 'request_id': 'req-53481889-cbd6-4aa7-9df4-9d882ee48b13'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.379469 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0160f2201546a385be13aea0454fdc', 'request_id': 'req-53481889-cbd6-4aa7-9df4-9d882ee48b13'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.398407 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] The token cache is being invalidate because project 9d0160f2201546a385be13aea0454fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:09:31.398788 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.399681 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.402503 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1451/2908] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:09:31 2022] DELETE /identity/v3/projects/9d0160f2201546a385be13aea0454fdc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:31.406846 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:31.409363 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:31.440485 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217806a-5577-497e-a5e7-a9df4a833e8e admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:31.472179 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:31.557438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:31.559719 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:31.560360 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:31.560869 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] PATH_INFO: `/v3/users/f928305bc4ac4ba4a35b0893d963ac8f` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:31.562859 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f928305bc4ac4ba4a35b0893d963ac8f)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:31.565535 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:31.665409 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f928305bc4ac4ba4a35b0893d963ac8f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.665820 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f928305bc4ac4ba4a35b0893d963ac8f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.694486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f928305bc4ac4ba4a35b0893d963ac8f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.732466 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f928305bc4ac4ba4a35b0893d963ac8f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.734056 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-29b30f36-e813-4ba7-a075-38f756e409af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f928305bc4ac4ba4a35b0893d963ac8f'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:31.770440 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1458/2909] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:09:31 2022] DELETE /identity/v3/users/f928305bc4ac4ba4a35b0893d963ac8f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:32.124067 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:32.126807 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:32.183304 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53481889-cbd6-4aa7-9df4-9d882ee48b13 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:32.273468 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:32.274125 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:32.274402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:32.274771 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] PATH_INFO: `/v3/projects/c1c777eae8de44b18cae19cd7d9d8cbc` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:32.276177 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c777eae8de44b18cae19cd7d9d8cbc)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:32.278626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:32.305399 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:32.354376 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c777eae8de44b18cae19cd7d9d8cbc', 'request_id': 'req-be613be4-b205-4103-b799-bdee9fe7c0ec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:32.362551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c777eae8de44b18cae19cd7d9d8cbc', 'request_id': 'req-be613be4-b205-4103-b799-bdee9fe7c0ec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:32.376350 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c777eae8de44b18cae19cd7d9d8cbc', 'request_id': 'req-be613be4-b205-4103-b799-bdee9fe7c0ec'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:32.397276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] The token cache is being invalidate because project c1c777eae8de44b18cae19cd7d9d8cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:09:32.397276 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:32.397745 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:32.399927 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1452/2910] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:09:32 2022] DELETE /identity/v3/projects/c1c777eae8de44b18cae19cd7d9d8cbc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:33.393534 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef85b7-249e-48ce-ac6a-67b9265e6dfc None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:33.394084 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef85b7-249e-48ce-ac6a-67b9265e6dfc None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:33.394211 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef85b7-249e-48ce-ac6a-67b9265e6dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:33.442999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-10ef85b7-249e-48ce-ac6a-67b9265e6dfc None None] MFA Rules not processed for user `2e323c9ad9b3491496d8023fed2731b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:33.444314 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-10ef85b7-249e-48ce-ac6a-67b9265e6dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:33.572779 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1459/2911] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:33.586686 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:33.588821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:33.693439 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be613be4-b205-4103-b799-bdee9fe7c0ec admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:33.698154 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10f80305dfe04303a9a9387535ed8ae4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:33.700558 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:33.701078 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:33.701687 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:33.704540 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:33.782008 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:33.784212 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:33.795758 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1453/2912] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:33.958131 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:33.958442 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:33.958776 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:34.016999 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:34.029257 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:34.030451 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:34.150979 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1460/2913] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:09:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:09:34.174095 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:34.175620 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:34.273275 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:34.315566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafb349-31db-4f10-8264-6e6a34b24746 service glance] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:34.321610 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:34.322249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:34.322529 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:34.322810 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:34.340018 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:34.341718 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:34.352313 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1454/2914] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:09:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:09:35.372004 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:35.375607 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:35.634301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51461c86-826d-4101-9de4-ad448d4c43e6 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:35.640008 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:35.640634 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:35.640913 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:35.641190 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:35.653410 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:35.673549 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:35.676982 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309d99-d515-4d79-aca4-fe136b97c891 service nova] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:35.701050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1461/2915] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:42.680694 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:42.683777 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:42.800627 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d03df-3fae-4a5e-9783-2efd3a4010cb service swift] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:42.814789 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:42.815566 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:42.815915 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:42.816315 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:42.818421 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:42.831361 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:42.834157 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:42.847829 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1455/2916] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:44.141022 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:44.141592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:44.141592 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:44.163424 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] MFA Rules not processed for user `f95b1dd078c842df97f8fb242e59020b`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:44.164633 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:44.171671 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1462/2917] 10.222.0.72 () {58 vars in 958 bytes} [Mon Sep 26 18:09:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:44.454412 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:44.463690 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af86c37-9199-4bb2-b57c-5c2b4cc24637 service neutron] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:44.467283 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:44.467875 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:44.468156 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:44.468429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:44.470164 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:44.484113 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:44.490737 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11c47c-84d1-4ddd-8110-6a9307727dc9 service neutron] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:44.502632 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1456/2918] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:44.853565 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:44.855196 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:44.960937 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca6f1e-4387-49c2-aedd-51cf1b5ad289 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:44.970945 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:44.971959 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:44.972810 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:44.973158 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:44.975104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:44.991374 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:44.993635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:45.016635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1463/2919] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:46.592438 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] REQUEST_METHOD: `POST` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:46.592958 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:46.593065 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:46.615328 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.auth.core [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:46.616574 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:46.624061 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1457/2920] 10.222.0.72 () {58 vars in 1009 bytes} [Mon Sep 26 18:09:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:46.637010 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:46.649165 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3dc12-63e3-4c72-888c-1961f6dd85a8 service placement] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:46.654137 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:46.654744 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:46.655058 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:46.655334 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:46.664516 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=100 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:46.666971 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:46.668579 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:46.682826 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1464/2921] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:09:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:09:56.610007 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=50 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:56.674759 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=2 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:06.326543 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:06.348198 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170e7b0-d15f-434a-baaf-21ae79a593b6 None None] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:06.357544 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:06.358611 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:06.359163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:06.359659 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:06.362956 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:06.387222 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:06.390654 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:06.409272 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1458/2922] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:10:16.402138 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:20.980783 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:20.982536 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:20.995822 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d26ea-2185-4c61-bf97-92290763525d service swift] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:20.999910 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b179c391ce574cd8836b43ea5bd94222', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:21.000641 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:21.001087 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:21.001493 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:21.013103 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:21.014635 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:21.027630 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1465/2923] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:10:23.183381 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:23.185851 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:23.203592 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7d288-07e5-4db8-a8de-162d8e820044 service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:23.212981 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1adfb9b41a1d4d31bccf5cc87b955526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:23.213781 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:23.214163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:23.214538 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:23.216594 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:23.233728 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:23.237780 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:23.250641 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1459/2924] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:10:31.021624 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=6 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:33.246685 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=6 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:38.590209 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:38.591705 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:38.664248 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b9118-e3c0-4301-9e7c-d41dceeca3e1 service neutron] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:38.669110 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:38.669839 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:38.670194 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:38.670486 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] PATH_INFO: `/v3/users/4f51a306feb34c33b3e9162d8253ae38` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:38.671712 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f51a306feb34c33b3e9162d8253ae38)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:38.674184 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:38.730572 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f51a306feb34c33b3e9162d8253ae38'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:38.731335 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f51a306feb34c33b3e9162d8253ae38'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:38.740449 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f51a306feb34c33b3e9162d8253ae38'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:38.755208 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f51a306feb34c33b3e9162d8253ae38'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:38.755864 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-87e3c044-1e25-43ac-a7f2-03e5669c0082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f51a306feb34c33b3e9162d8253ae38'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:38.774048 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1466/2925] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:10:38 2022] DELETE /identity/v3/users/4f51a306feb34c33b3e9162d8253ae38 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:10:38.968858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:38.992129 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7002d26-efaa-4d35-8763-df61884565aa service nova] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:39.073739 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:39.074400 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:39.074657 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:39.074969 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] PATH_INFO: `/v3/projects/21586a671de94dc9b7b73f5dbaac6a01` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:39.076130 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21586a671de94dc9b7b73f5dbaac6a01)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:39.078027 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:39.133058 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21586a671de94dc9b7b73f5dbaac6a01', 'request_id': 'req-95707a59-21d7-41d1-a3e9-edc8c81c5167'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:39.145249 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21586a671de94dc9b7b73f5dbaac6a01', 'request_id': 'req-95707a59-21d7-41d1-a3e9-edc8c81c5167'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:39.153454 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21586a671de94dc9b7b73f5dbaac6a01', 'request_id': 'req-95707a59-21d7-41d1-a3e9-edc8c81c5167'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:39.172949 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] The token cache is being invalidate because project 21586a671de94dc9b7b73f5dbaac6a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:10:39.173942 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:39.174993 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:39.178031 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1460/2926] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:10:38 2022] DELETE /identity/v3/projects/21586a671de94dc9b7b73f5dbaac6a01 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:10:48.768322 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:49.167885 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:35.558526 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d789b8-0767-4417-965e-fe7370633351 None None] REQUEST_METHOD: `POST` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:35.559113 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d789b8-0767-4417-965e-fe7370633351 None None] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:35.559733 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d789b8-0767-4417-965e-fe7370633351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:35.581202 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.auth.core [None req-36d789b8-0767-4417-965e-fe7370633351 None None] MFA Rules not processed for user `1de6964e61bd46eca46e1db919ae14c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:35.582651 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-36d789b8-0767-4417-965e-fe7370633351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:35.640116 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1467/2927] 10.222.0.72 () {58 vars in 967 bytes} [Mon Sep 26 18:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:35.687637 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:35.689841 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:35.764050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95707a59-21d7-41d1-a3e9-edc8c81c5167 admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:35.769736 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ba853553814d49bf3c3a92c29fd27f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:35.770344 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] REQUEST_METHOD: `GET` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:35.770626 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:35.770928 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:35.779163 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:35.780776 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:35.789612 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1461/2928] 10.222.0.72 () {60 vars in 1384 bytes} [Mon Sep 26 18:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:11:42.891377 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d789b8-0767-4417-965e-fe7370633351 None None] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:42.892927 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-36d789b8-0767-4417-965e-fe7370633351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:42.967887 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d789b8-0767-4417-965e-fe7370633351 None None] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:42.973301 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:42.973942 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:42.974228 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:42.974504 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] PATH_INFO: `/v3/users/f95b1dd078c842df97f8fb242e59020b` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:42.975683 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95b1dd078c842df97f8fb242e59020b)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:42.977663 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.026436 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95b1dd078c842df97f8fb242e59020b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.027086 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95b1dd078c842df97f8fb242e59020b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.035405 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95b1dd078c842df97f8fb242e59020b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.047250 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95b1dd078c842df97f8fb242e59020b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.047972 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95b1dd078c842df97f8fb242e59020b'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.062769 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1468/2929] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:42 2022] DELETE /identity/v3/users/f95b1dd078c842df97f8fb242e59020b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:43.240600 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.251947 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153980a2-0859-48b9-b5b9-7bdf00b296d7 service swift] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.313305 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.313858 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.314149 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.314429 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] PATH_INFO: `/v3/projects/c747c13a887a4f45bf8a940833be3831` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.316331 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c747c13a887a4f45bf8a940833be3831)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.318280 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.373821 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c747c13a887a4f45bf8a940833be3831', 'request_id': 'req-71bb097f-16cb-429d-8bb5-c5900123e0df'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.380832 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c747c13a887a4f45bf8a940833be3831', 'request_id': 'req-71bb097f-16cb-429d-8bb5-c5900123e0df'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.389050 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c747c13a887a4f45bf8a940833be3831', 'request_id': 'req-71bb097f-16cb-429d-8bb5-c5900123e0df'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.406462 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] The token cache is being invalidate because project c747c13a887a4f45bf8a940833be3831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:43.406752 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.407444 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.413569 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1462/2930] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:43 2022] DELETE /identity/v3/projects/c747c13a887a4f45bf8a940833be3831 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:43.414554 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.415911 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:43.430574 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982808-610d-4595-9a71-3d1445036f4f admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.451402 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:43.492100 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.492780 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.493067 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.493371 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] PATH_INFO: `/v3/users/2e323c9ad9b3491496d8023fed2731b4` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.494591 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e323c9ad9b3491496d8023fed2731b4)` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.496669 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.554995 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e323c9ad9b3491496d8023fed2731b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.555654 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e323c9ad9b3491496d8023fed2731b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.569412 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e323c9ad9b3491496d8023fed2731b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.586275 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e323c9ad9b3491496d8023fed2731b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.587050 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.notifications [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e323c9ad9b3491496d8023fed2731b4'} {{(pid=81409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.603898 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1469/2931] 10.222.0.72 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:43 2022] DELETE /identity/v3/users/2e323c9ad9b3491496d8023fed2731b4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:43.810119 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Authenticating user token {{(pid=81410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.812311 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.fernet_utils [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:43.834886 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb097f-16cb-429d-8bb5-c5900123e0df admin admin] Validating token access rules against request {{(pid=81410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.861131 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:43.923568 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91cdfaf440314ad2b1f94da968a2f0eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bbe873ec2e4b4dc7b12dd87b247ac3dc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.924232 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.924551 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.925491 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] PATH_INFO: `/v3/projects/99ac71f2187a4db1be4f404fa3cf04a5` {{(pid=81410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.926720 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ac71f2187a4db1be4f404fa3cf04a5)` {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.928755 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] RBAC: Authorization granted {{(pid=81410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.985343 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ac71f2187a4db1be4f404fa3cf04a5', 'request_id': 'req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.993394 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ac71f2187a4db1be4f404fa3cf04a5', 'request_id': 'req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:44.003582 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ac71f2187a4db1be4f404fa3cf04a5', 'request_id': 'req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451'} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:44.017402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] The token cache is being invalidate because project 99ac71f2187a4db1be4f404fa3cf04a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:44.017402 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:44.017863 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG keystone.notifications [None req-e70d2f6d-db8f-4dd1-bd6e-5bb540c8d451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:44.023269 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: [pid: 81410|app: 0|req: 1463/2932] 10.222.0.72 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:43 2022] DELETE /identity/v3/projects/99ac71f2187a4db1be4f404fa3cf04a5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:53.599038 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:54.013218 dv-cinder-859318-1-iscsi devstack@keystone.service[81410]: DEBUG dbcounter [-] [81410] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:13.154865 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Authenticating user token {{(pid=81409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:13.156761 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:13.230438 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33242f6-a878-46d9-82b1-e7f5fb2453bd admin admin] Validating token access rules against request {{(pid=81409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:13.307966 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92937af60f80421094f4bdf6c7620283', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54e4118bb98b467c9b698a23bba22007', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:13.308494 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] REQUEST_METHOD: `GET` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:13.308862 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] SCRIPT_NAME: `/identity` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:13.309231 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:13.311081 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.fernet_utils [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:13.323753 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:13.325471 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f639863-5ad1-42a0-8c7f-840f663ef78d service placement] RBAC: Authorization granted {{(pid=81409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:13.336672 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: [pid: 81409|app: 0|req: 1470/2933] 10.222.0.72 () {60 vars in 1365 bytes} [Mon Sep 26 18:12:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:12:23.332104 dv-cinder-859318-1-iscsi devstack@keystone.service[81409]: DEBUG dbcounter [-] [81409] Writing DB stats keystone:SELECT=58 {{(pid=81409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}